jerk Banned
#1

I've been messing about with shell scripts the last few days and I'm fairly ****e but hey...

Anyway, why don't you stick up your best shell scripts that make monotonous tasks quick, dead handy scripts, whatever...

If it isn't included in linux and you or a friend wrote it stick it up and share the wealth...

BenH Registered User
#2

This might be of some interest, especially if your a redhat fan:

http://night-shade.org.uk/basic-configs/

Hecate Registered User
#3

I wrote this to automatically update the ports tree on my freebsd box; it has a cron entry to run every saturday morning, gotta keep up to date with the latest and greatest:


#!/bin/sh

IFTEST=`ifconfig tun0 |grep UP`

if [ -n "$IFTEST" ] ; then

/usr/local/bin/cvsup -g -L 2 /root/ports-supfile

else

ppp -ddial nolimits
sleep 40
/usr/local/bin/cvsup -g -L 2 /root/ports-supfile
killall ppp
fi


The box also stores a fair amount of divxs at any one time, so I have a script to search out any movies that are on the server and output them to a html page:


#!/bin/sh

echo "<html><h1>Movie feed .. yum</h1><br><p align=left><pre>" > /usr/local/www/data/film_listing.htm

find / -name '*.avi' >> /usr/local/www/data/film_listing.htm
find / -name '*.ogm' >> /usr/local/www/data/film_listing.htm

echo "</p></pre></html>" >> /usr/local/www/data/film_listing.htm


Thats it for now, apart from a few one liners that do boring but useful stuff like dial up with ppp and set the time on the server using ntpdate.

I had some pretty handy ones for backing up users home directories and mysql/pgsql accounts to tapes using a combination of tar and rsync on the DIT netsoc server, but it's offline at the moment

1 person has thanked this post
parasite Registered User
#4

i've been googling for ways to total the amount of hours of my ppp sessions per month, but ntop etc seem like overkill, and i've been trying to think of a script to write for it, what's the best way to go about it, grep ps for pp every so often or something better ?

ssh Registered User
#5

Get your boys out of there, this is gonna be a big one... Adds services from your current runlevel under debian...


#!/bin/sh
                                                                                
echo addrc Enable services at startup for Debian systems
function print_current_services
{
        \
ls /etc/rc$rl.d/ |
        while 
read line; do
                
order=$(echo $line cut -b 2,3)
                echo 
$order: $(echo $line sed -'s/S[0-9]*//g')
        
done
}
                                                                                
rl=$(runlevel cut -" " -f 2)
service=XX
                                                                                
while [ $service == XX ]; do
        echo -
"Enter the name of the service [Hit return to see a list]: "
        
read service
                                                                                
        tmpvar
=XX$service
                                                                                
        
if [ $tmpvar == XX ]; then
                service
=XX
                
echo
                \
ls /etc/init.d/
                echo
        
fi
                                                                                
        
\ls /etc/init.d/ | egrep -^$service$
        if [ $? != 
] && [ $tmpvar != XX ]; then
                
echo Service does not exist
                service
=XX
        fi
done
                                                                                
echo -n Checking to see if $service is already configured to run...
\
ls /etc/rc$rl.sed -'s/S[0-9]*//g' grep -^$service$ > /dev/null
if [ $? == ]; then
        
echo Failed already there
        
exit
fi
echo passed
                                                                                
echo "Here is a list of services and the order in which they start: "
echo
print_current_services
                                                                                
sorder
=XX
while [ $sorder == XX ]; do
        echo
        echo -
"Enter the order you want the service to start in (10-99: "
        
read sorder
        
echo $sorder egrep -^[0-9][0-9]$
        if [ $? != 
]; then
                
echo Enter a number between 10 and 99
                sorder
=XX
        fi
done
                                                                                
echo -n Adding service to list...
ln -/etc/init.d/$service /etc/rc$rl.d/S$sorder$service
                                                                                
echo Checking to see if the service is listed for shutdown...
\
ls /etc/rc1.d/ | sed -'s/K[0-9]*//g' egrep -^$service$
if [ $? != 
]; then
        ln 
-/etc/init.d/$service /etc/rc1.d/K$sorder$service
fi
echo done
                                                                                
echo "Here's how things look now:"
 
print_current_services

flamegrill Beardy Mod
#6

here's a vhost adding script I fired togeter, its nasty but it works. Basically it adds the proper folders etc for the user/site in question. It prints the httpd.conf entry to screen and you can do what you like with that.

I also wrote another version that does some fancy log work with cronolog. I'll leave that for another day.

I've got a nice bit of c that I'll paste in a bit to check /var/log/messages for time online




[COLOR=silver]#!/usr/bin/php -q[/COLOR]
<?

if (isset($argv[1]))
        {

        
$domain =  $argv[1];
    
$user $argv[2];
    if (isset(
$argv[3]))
        { 
        
$vhostip $argv[3];    
        }
    else 
        {
        
$vhostip "217.114.170.92";
        }
$vhost "
<VirtualHost 
$vhostip>
ServerName [url]www.[/url]
$domain
ServerAdmin 
$user@$domain
DocumentRoot /home/
$user/$domain/web
ServerAlias 
$domain
DirectoryIndex index.html index.htm index.php index.php4 index.php3 index.shtml index.cgi index.pl
ScriptAlias  /cgi-bin/ /home/
$user/$domain/cgi-bin/
AddHandler cgi-script .cgi
AddHandler cgi-script .pl
ErrorLog /home/
$user/$domain/logs/error_log
CustomLog /home/
$user/$domain/logs/access_log combined
AddType application/x-httpd-php .php .php4 .php3
AddType text/html .shtml
AddHandler server-parsed .shtml
#ErrorDocument 400 /error/invalidSyntax.html
#ErrorDocument 401 /error/authorizationRequired.html
#ErrorDocument 403 /error/forbidden.html
#ErrorDocument 404 /error/fileNotFound.html
#ErrorDocument 405 /error/methodNotAllowed.html
#ErrorDocument 500 /error/internalServerError.html
#ErrorDocument 503 /error/overloaded.htm
</VirtualHost>
"
;

print 
$vhost;

// nasty shell code follows to make directories to keep apache happy :)
$shellcode "mkdir -p /home/$user/$domain/web; mkdir -p /home/$user/$domain/cgi-bin/; mkdir -p /home/$user/$domain/logs/ ";
#passthru($shellcode);
}

else
        {
        print 
"Useage is as follows:\n\n";
        print 
"vhostadd.php <domainname> <user> optionally give vhost-ip (without the <>)\n";

        }
?>


dahamsta Banned
#7

[EDIT: Ah, because of the domain. Tool. Never mind.]

You can lose the multiple mkdir's, it accepts multiple directories.

mkdir -p /home/$user/$domain/web /home/$user/$domain/cgi-bin /home/$user/$domain/logs
HTH,
adam

1 person has thanked this post
flamegrill Beardy Mod
#8

Originally posted by dahamsta
[EDIT: Ah, because of the domain. Tool. Never mind.]

You can lose the multiple mkdir's, it accepts multiple directories.

bWtkaXIgLXAgL2hvbWUvJHVzZXIvJGRvbWFpbi93ZWIgL2hvbWUvJHVzZXIvJGRvbWFpbi9jZ2ktYmluIC9ob21lLyR1c2VyLyRkb21haW4vbG9ncw==HTH,
adam


Ah yes indeedy, a less evil shell argument

Still looking for a powercable for the old box with the pppd timer thing.

flamegrill Beardy Mod
#9

This isn't 100% accurate, but it will be good up to 1-2 hours of your actual usage. Saved me heaps on UTV in the later months




/*
    program to parse /var/log/messages for pppd dialup information
    by Paul Kelly (paul@dahomelands.net) 2002
*/


#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define DATELEN 16


typedef struct ppp_stat_st {
    
struct {
        
unsigned long connectsdisconnects;
        
unsigned long connected_secs;
        
unsigned long sentkrcvdk;
    } 
total;

    
struct {
        
unsigned long hupterm;
    } 
reasons;

    
struct {
        
unsigned long connected_secs;
        
unsigned long sentkrcvdk;
    } 
best;
pstat;


void pppd_log_stat(pstat *psFILE *fp) {
    
char buf[BUFSIZ], *p, *q;
    
unsigned long l;


    while (
fgets(bufsizeof buffp)) {
        
buf[sizeof buf 1] = 0;

        if (! (
strstr(buf DATELEN"pppd"))) /* not a pppd logline */
            
continue;

        if (
strstr(buf"LCP Echo")) /* debug mode on :[ */
            
continue;

        if (
strstr(p"Connection terminated.")) {
            
ps->total.disconnects ++;
            continue;
        }

        if (
strstr(p"SIGHUP")) {
            
ps->reasons.hup++;
            continue;
        }

        if (
strstr(p"signal 15.")) {
            
ps->reasons.term++;
            continue;
        }

        if (
strstr(p"Connect time")) {
            
strtoul(q+13010) * 60;
            
ps->total.connected_secs += l;

            if (
ps->best.connected_secs l)
                
ps->best.connected_secs l;

            continue;
        }

        if (
strstr(p"Sent ")) {
            
strtoul(q+5NULL10) / 1024;
            
ps->total.sentk += l;

            if (
ps->best.sentk l)
                
ps->best.sentk l;

            if (! (
strstr(p"ived")))
                continue; 
/* should never happen */

            
strtoul(q+5NULL10) / 1024;
            if (
ps->best.rcvdk l)
                
ps->best.rcvdk l;
            
ps->total.rcvdk += l;
            continue;
        }
    }
}


void paul_kelly_fmt_print(FILE *fppstat *ps) {
    
fprintf(fp,
        
"PPP INTERFACE USAGE REPORT (complete)\n"
        "\n"
        "%13lu ppp sessions\n"
        "%13lu Hours online\n"
        "%13lu Kilobytes sent\n"
        "%13lu Kilobytes received\n"
        "\n"
        "Averages for this period:\n"
        "\n"
        "%13lu Hours online\n"
        "%13lu Kilobytes sent\n"
        "%13lu Kilobytes received\n"
        "\n"
        "Best records for a single session:\n"
        "\n"
        "%13lu Hours online\n"
        "%13lu Kilobytes sent\n"
        "%13lu Kilobytes received\n"
        "\n"
        "Connection termination reason counts:\n"
        "\n"
        "%13lu Modem hangup\n"
        "%13lu Received sigterm\n"
        "\n"
,

        
ps->total.disconnects,
        
ps->total.connected_secs 60 60,
        
ps->total.sentk,
        
ps->total.rcvdk,

        
ps->total.connected_secs 60 60 ps->total.disconnects,
        
ps->total.sentk ps->total.disconnects,
        
ps->total.rcvdk ps->total.disconnects,

        
ps->best.connected_secs 60 60,
        
ps->best.sentk,
        
ps->best.rcvdk,

        
ps->reasons.hup,
        
ps->reasons.term
    
);
}


int main(int acchar **av) {
    
pstat ps;
    
char *filename "/var/log/messages";
    
FILE *fp;

    if (
ac 1)
        
filename av[1];

    if (! (
fp fopen(filename"r"))) {
        
perror(filename);
        return 
EXIT_FAILURE;
    }

    
memset(&ps0sizeof ps);
    
pppd_log_stat(&psfp);

    
fclose(fp);

    
paul_kelly_fmt_print(stdout, &ps);
    return 
EXIT_SUCCESS;
}



If it won't copy/paste compile from here, i'll email people the code. :-)

Currently writing a few bits n bobs for handy administration , will post them if/when they ever get completed.

jerk Banned
#10

nice one lads, the vhost one will be dead handy.

ssh Registered User
#11

Analyses squid logs... needs python-dns too. A little inaccurate also because it should divide by 1024, not 1000...


#!/usr/bin/python
                                                                                                                                                            
import DNS
import re
import sys
                                                                                                                                                            
DNS
.ParseResolvConf()
requests = {}
results = {}
hitsperhost = {}
                                                                                                                                                            
host ''
size 0
type 
''
url ''
                                                                                                                                                            
biggest 0
biggesthost 
''
biggesturl ''
                                                                                                                                                            
displayedsummary 0
                                                                                                                                                            
deflog 
'/var/log/squid/access.log'
                                                                                                                                                            
try:
        
lf sys.argv[1]
except:
        
lf deflog
                                                                                                                                                            
try:
        
open(lf "r")
except IOError:
        print 
"Could not open log file"
        
sys.exit(1)
                                                                                                                                                            
while 
0:
        
line f.readline()
                                                                                                                                                            
        if 
line == '':
                break
                                                                                                                                                            
        
splitline line.split()
                                                                                                                                                            
        
host splitline[0]
        
size int(splitline[9])
        
type splitline[10]
        
url splitline[6]
                                                                                                                                                            
        if 
displayedsummary == 0:
                print 
'First access in log at   ',splitline[3],splitline[4]
                
displayedsummary 1
        
try:
                
requests[host] = requests[host] + size
                hitsperhost
[host] = hitsperhost[host] + 1
        except KeyError
:
                try:
                        
requests[host] = size
                        hitsperhost
[host] = 1
                except
:
                        print 
"You don't seem to have emulate_httpd_log enabled"
                        
sys.exit(1)
                                                                                                                                                            
        if 
size biggest:
                
biggest size
                biggesthost 
host
                biggesturl 
url
                                                                                                                                                            
        
try:
                
results[type] = results[type] + size
        except KeyError
:
                try:
                        
results[type] = size
                except
:
                        print 
"You don't seem to have emulate_httpd_log enabled"
                        
sys.exit(1)
 
try:
        print 
'Last access in log at    ',splitline[3],splitline[4]
except NameError:
        print 
'Log file is empty'
        
sys.exit(1)
 
print 
'Statistics by Host:'
for i in requests.keys():
        try:
                
hostname DNS.revlookup(i)
        
except:
                
hostnamei
        
print (requests[i] / 1000),'KB by',hostname,'in',hitsperhost[i],'requests'
 
hits 0
misses 
0
rehit 
re.compile('.*HIT.*')
 
print 
'\nStatistics by request result:'
for i in results.keys():
        if 
rehit.match(i):
                
hits hits results[i]
        else:
                
misses misses results[i]
 
        print (
results[i] / 1000),'KB           marked as',i
 
percentagehits 
float(hits) / float(misses hits) * 100
 
print 'Total Hits =     ',hits 1000,'KB'
print 'Total Misses =   ',misses  1000,'KB'
print '% Hits =         ',percentagehits,'%'
 
print 'Biggest download was',biggesturl,'by',DNS.revlookup(biggesthost),'at',biggest 1000,'KB'

flamegrill Beardy Mod
#12

I've decided to make this a sticky as it will always be handy for n00bs

Paul

charlieroot Registered User
#13

Originally posted by Hecate
I wrote this to automatically update the ports tree on my freebsd box; it has a cron entry to run every saturday morning, gotta keep up to date with the latest and greatest:



You might want to put a pkgdb -u or pkgdb -fu in there somewhere otherwise you'll end up with some nasty problems.

Noel.

nadir Registered User
#14

erm, im kinda unsure what i can or cant put here, I have loads of small simple scripts ive thrown together but erm, im not sure if u could consider them all bash scripts, like i have a nice little xmms-irssi script that i rewrote from the irssi page with some updates and stuff like that. This might be way off topic, here is a very simple easy to follow firewall script, im not sure how secure this is but it functions fairly well against port scanners, allows internal nat, ip forwarding, masquarding, filteres tcp and udp ports, (allows UT2003, Quake3, Army Ops Recon and others). Basically it will allow outbound connections but not inbound and still allow routing from an internel network, in otherwords u should be able to use all ur fav apps, like irc(DCC), p2p, games web .etc
------------------------------------------------------------------------------------------------

#simple firewall, by nadir - 2003
echo "loading irc modules"
modprobe ip_conntrack_irc ports=$IRC_PORTS || SYSLOG
modprobe ip_nat_irc
#set vars
UNPRIVPORTS="30000:35000" # unprivileged port range

# Remove any existing rules from all chains
echo "flushing previous rules"
iptables --flush
iptables -t nat --flush
iptables -t mangle --flush

echo "Starting nadirs firewall"
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -A INPUT -d 10.0.0.0/8 -i ppp0 -j DROP
iptables -A INPUT -m state --state NEW,INVALID -j DROP
iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE
iptables -A INPUT -p tcp --syn -j DROP

#udp and tcp ports to allow.

iptables -I INPUT 1 -p tcp -m multiport --dport 113,20045,9201,8481,8888 -j ACCEPT

iptables -I INPUT 1 -p udp -m multiport --dport 1716,1717,1718,8777,27900 -j ACCEPT

#This enables masquarding
iptables -A POSTROUTING -t nat -o eth1 -j MASQUERADE

# activate IP-Forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward

echo "firewall rules implemented"
echo "----------------------Rules are-----------------------"
#list rules
iptables -t nat -nL
iptables --list

Typedef Registered User
#15

echo "Welcome to the ftp user addition utility"
echo "You can cancel this process at any time by"
echo "Holding down CRTL and tapping the c key"
echo " "
echo "Please enter the username you'd like to create and press [ENTER]"
read luser

echo "User " $luser " Ok y/n?"
read val

if [ "$val" == "n" ]; then
echo "Not continuing "
exit -1
elif [ "$val" == "N" ]; then
echo "Not continuing "
exit -1
elif [ "$val" == "y" ]; then
mkdir /usr/data/$luser
/usr/sbin/useradd -d /usr/data/$luser -s /sbin/nologin $luser
passwd $luser
passwd -x 2 $luser
/usr/sbin/usermod -G $luser,writers $luser
/bin/chown $luser /usr/data/$luser
/bin/chgrp writers /usr/data/$luser
/bin/chmod g+w /usr/data/$luser
/bin/chmod 777 /usr/data/$luser
elif [ "$val" == "Y" ]; then
mkdir /usr/data/$luser
/usr/sbin/useradd -d /usr/data/$luser -s /sbin/nologin $luser
passwd $luser
passwd -x 2 $luser
/usr/sbin/usermod -G $luser,writers $luser
/bin/chown $luser /usr/data/$luser
/bin/chgrp writers /usr/data/$luser
/bin/chmod g+w /usr/data/$luser
/bin/chmod 777 /usr/data/$luser
else
echo "You have not confirmned nor denied this input, exiting now"
exit 0
fi

------------------------------------------------------------------

for file in $( ls -t *.mp3 *.ogg | head -250 | tr ' ' '_');
do
#scp $file root@10.5.2.113:/storage/mp3
scp -C "`echo $file | sed 's/_/ /g'`" root@10.5.4.0:/storage/mp3
done


------------------------------------------------------------------

/data/pgsql-site-1.5/bin/vacuumdb -p 6543 -z stockbyte-dev;
/data/pgsql-site-1.5/bin/pg_dump -p 6543 stockbyte-dev > /data1/site-15-db-backup/$1;
/bin/rm /data1/site-15-db-backup/$1.bz2;
/usr/bin/bzip2 /data1/site-15-db-backup/$1;

Want to share your thoughts?

Login here to discuss!