Urchin

From Rackerbox Wiki

Jump to: navigation, search

Information on Urchin Administration


http://www.google.com/analytics/urchin_downloads.html


Contents

Urchin Installation Guide

  1. Log in to remote server where Urchin is to be installed (SSH)
  2. Create a directory to store the urchin installer files 
  5. Extract the urchin application (e.g. sh urchin5600_redhat72.sh). This will self extract all of the contents of Urchin.
  6. Run the installer (./install.sh)
  7. You can leave pretty much everything the default values, except for the following:
         * change the port number from 9999 to 10000
         * change the group to psaserv if the customer is running Plesk 6 or higher 
  8. Modify the init script to point to the right directory (change value of INSTALLDIR)
  9. Install the init script (chmod +x urchin.init; mv urchin.init /etc/init.d/urchin; chkconfig --add urchin)
 10. Start up Urchin if it's not already running (/etc/init.d/urchin start)
 11. Obtain a valid license key from https://secure.urchin.com/var/login
 12. After logging in, click on Get Licenses. Choose Urchin5; continue; Then pick the quantity. Note that quantity is in denomations of 100, so 2 would be 200. Finally, complete the purchase. Save the key into your clipboard buffer.
 13. At this point, login to the customer's Urchin (http://IP:10000). The default login will be: admin/urchin
 14. When prompted for the key, paste in the one earlier, obtained from Urchin's website. You can use the default values during the installation process. 
 15. Finally, use the pre-fab response for Urchin when filling out the ticket. Make sure you alter the comment up a little bit as some of the remarks may not be relevant.

Urchin Migration Instructions

The Urchin migration is an easy process. You can find the official migration documentation at:

http://www.google.com/support/urchin45/bin/answer.py?answer=28511

Here are the basics:

  1. Backup configuration (old server)
    /usr/local/urchin/util/uconf-export > /root/urchin_conf.txt
    Stop Urchin (both servers)
    /etc/init.d/urchin stop
    Change the FTP Log Source from the public IP to the private IP (old server)
    sed 's/67.192.49.7/192.168.1.7/g' /root/urchin_conf.txt > /root/urchin_conf.new.txt
    Transfer /root/urchin_conf.new.txt to new server
    Import configuration (new server) If importing over 100 profiles, you must remove the <Global> section to prevent the license being reset
    uconf-import -o -f /root/urchin_conf.new.txt
    Transfer data from old server to new server using whatever method you are comfortable with (such as tar,scp,rsync)
    /usr/local/urchin/data
    Fix permissions (new server)
    chown -R urchin:urchin /usr/local/urchin/
    Start Urchin on new server
    /etc/init.d/urchin start

Urchin tips and suggestions

Standard

  • log each domains access log in separate file(this makes configuration simpler as you dont have to configure log filters)
  • configure logroate(this allows Urchin to process smaller files, work more effeciently, and prevent conflicts if data in log files is older than what Urchin has already archived data for)
  • don't compress log files before Urchin processes them(while Urchin can read compressed log files, it takes much more processing power and time to process)
  • schedule processing to occur at different times of the day during off-peak hours for each profile
  • don't restart Urchin services or server during times in which log files are being processed

Utilizing these tips and strategies will help ensure Urchin runs efficently and helps prevent issues down the road.


PLESK

  • Log each domains access log in separate file. This makes configuration simpler as you don't have to configure log filters(Plesk automatically does this by default, so you don't have to worry about this)
  • Configure log rotation for each log files. This allows Urchin to process smaller files, work more efficiently, and prevent conflicts if data in log files is older than what Urchin has already archived data for. (you can set this up within Plesk under "Log Manager" in each domain)
  • Don't compress log files before Urchin processes them. While Urchin can read compressed log files, it takes much more processing power and time to process.
  • Schedule processing to occur at different times of the day during off-peak hours for each profile.
  • Don't restart Urchin services or server during times in which log files are being processed. Doing so can leave processing hung and prevent future automatic log processing.

Utilizing these tips and strategies will help ensure Urchin runs efficiently and helps prevent issues down the road.


Manually run processing

You can run processing manually for a domain, and even a particular date. However, the data files will get created as the user you are executing the script as.

Option 1: Run as urchin user

su urchin
cd /usr/local/urchin/bin
./urchin -p www.theblahblah.com

Option 2: Change permissions back to urchin afterwards

cd /usr/local/urchin/bin
./urchin -p www.theblahblah.com -d 20080507
./urchin -p www.theblahblah.com -d 20080508

cd ../data
chown -r urchin:urchin *

Troubleshooting Points to note

  • Sometimes it will report that the scheduler is not running. Verify before you logoff that the urchind process is running in addition to urchinwebd. Urchind is the scheduler. If it is not running, run :
cd /usr/local/urchin; ./bin/urchind;
It should then run subsequently on future Urchin restarts.
  • If the license is corrupted or needs to be reinstalled, you can run the following command to zero out the license:
<URCHIN_DIR>/util/uconf-driver action=set_parameter recnum=1 ct_serial=0 uconf-driver action=set_parameter recnum=1 ct_license=0


---Urchin - urchin change password---

cd /usr/local/urchin4/utils

./uconf-driver action=set_parameter table=user name="(admin)" ct_password=urchin



Unable to open database for writing since it has been archived.

Turn off Auto Archive

Login to the server and

/usr/local/urchin/data/reports/<profile>
ls -1 *archive.zip | xargs -iXX unzip XX
rm -f *archive.zip
chown urchin:urchin * (or whatever ownership the other files in directory have)

Re-run the profile.


      1. Urchin's fix for the Cross Site Scripting (XSS) Vulnerability

http://www.google.com/support/urchin45/bin/answer.py?answer=76399&topic=7395

UTM Troubleshooting

http://www.google.com/support/urchin45/bin/answer.py?answer=28695&topic=7394

that

• Make certain there are only 4 variables before \"%r\" in the Apache LogFormat for the domain

• Make sure that the value for "UTM domain" on the Reporting tab matches the value used for the "_udn" variable in the __utm.js file in the document root of the website. If set to auto, UTM domain blank

• No "Client Parameters" being reported: Verify that the __utm.js file in document root matches the UTM-2 __utm.js file in /usr/local/urchin/util/utm


Updaing permissions

1. Shutdown Urchin

2. Update /usr/local/urchin/var/urchinwebd.conf and

/usr/local/urchin/var/urchinwebd.conf.template. Set Group parameter to

"psaserv"

3. Run /user/local/urchin/util/inspector -r

This will update all permissions according to the urchinwebd.conf file.

4. Start Urchin

Log file path for Plesk to grab ssl logs as well:

/statistics/logs/access(_ssl)?_log.processed*



Useful uconf-driver commands

http://www.google.com/support/urchin45/bin/answer.py?answer=28788&topic=7400


-This recnum is the number listed after successful comands, such as clearing out pending tasks, it is also the number used for temporary remote log files.

/usr/local/urchin/util/uconf-driver action=get recnum=120


-Get current values for a profile

/usr/local/urchin/util/uconf-driver action=get table=profile name=profilename.com


-Updating parameters. All parameters listed at http://www.google.com/support/urchin45/bin/answer.py?answer=28412

This one turns off archiving for a specific profile.

/usr/local/urchin/util/uconf-driver action=set_parameter table=profile name=$profile cr_archivedata=off


-Adding a new user "rack" (good for if password is unknown)

./uconf-driver action=add table=user name=rack ct_name=rack ct_password=PASSWORD cs_adminlevel="1" cs_language="en" cs_region="us" ct_fullname="Rackspace Urchin User"



      1. Urchin wrapper to be run from logrotate


    1. Script to process logs right before rotation
#!/bin/sh

if [ -f /var/lock/subsys/urchinrotate ]
then
        echo "" > /dev/null
else
        touch /var/lock/subsys/urchinrotate
        for i in `ls /usr/local/urchin/data/reports`
        do
                /usr/local/urchin/bin/urchin -p $i > /dev/null 2>&1
        done
fi



    1. Logrotate config
var/log/httpd/*log {
    rotate 10
    weekly
    prerotate
        /usr/local/bin/urchinLogrotate.sh >/dev/null 2>&1
    endscript
    missingok
    notifempty
    sharedscripts
    postrotate
        /bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true
        rm -f /var/lock/subsys/urchinrotate
    endscript
}



      1. Interesting log file path wildcards

http://www.google.com/support/urchin45/bin/answer.py?answer=28776



      1. Google Analytics and Urchin 5 UTM at the same time###

http://www.google.com/support/analytics/bin/answer.py?hl=en&answer=74937



      1. Statistics Discrepancies ###



        1. Errors####


Authentication Error - can't grab global config from db. (6006) Please log-in again.


(/usr/local/urchin/util) # ./uconf-driver action=get table=global


ERROR: (7004-557-91) Unable to open database for writing - check permissions.

DETAIL: Permission denied


    1. Filter Field##

Some filter fields are automatically generated by Urchin. Easiest way to tell what filed to use to filter results out of a particular report is to look at the URL:

http://www.google.com/support/urchin45/bin/answer.py?answer=28466&topic=7380



    1. Urchin licensing###

With an Urchin license for 100, 200 or more domains, you can setup that many profiles, but can only schedule n-1. This is due to the __domaindb(geo-conf) scheduled task that occupies one spot.

Personal tools