PDA

View Full Version : Mirroring Graphs To Reduce Load




Ronin
12-15-2007, 02:39 AM
Everyone,

I remember on 11/5 ronpaulgraphs.com got hammered from everyone wanting to look at the charts. I propose people volunteer mirroring.

I wrote a simple script below that can be run on Unix or cygwin servers. This will download the daily graph every 5 minutes. Nothing fancy. I don't have a server at the moment that I can make public, but others feel free to use it. If you don't have wget you might try curl or lwp-download (perl).

Please reply with a reply with a URL if you mirror so everyone knows. I propose a sticky.

Commands to run:


$ nohup ./get_graph.sh &
$ exit

Keep in mind this will continue to run in the background until it is killed!! Leave out the nohup if you want it in the foreground.

------ script -----

#!/bin/bash

# edit globals to your liking

#PATH=$PATH:/usr/local/bin

URL="http://www.ronpaulgraphs.com/yesterday_vs_today_line.png"
IMG=`basename $URL`
PUBLIC_HTML=/home/foobar/public_html
WGET="wget -q -O $IMG.$$ $URL"
CURL="curl -so $IMG.$$ $URL"

cd $PUBLIC_HTML

while [ 1 -eq 1 ]; do

$WGET && mv $IMG.$$ $IMG || rm $IMG.$$ 2>/dev/null

sleep 300
done

-- end

PS: Maybe someone could attach some html that will refresh this image every 5 minutes.

Edit: Added curl and error correction for wget/curl

Dan D.
12-15-2007, 03:57 AM
Unfortunately, I think if I was to mirror, my poor site would use up all its bandwidth in an hour.

Great idea though, with some caveats:
More people have curl than wget installed, so you might want to provide a script for that.
Don't alter other people's paths, it's impolite. If a Unix admin is installing this, they know where their own curl or wget is (even if they don't know how to use it) and can modify the script accordingly.

Kacela
12-15-2007, 08:25 AM
Don't forget http://paulcash.slact.net/ !

That's where I ended up going on 11/5 when http://www.ronpaulgraphs.com/ was overloaded... :)

HelpRonPaulWin
12-15-2007, 01:15 PM
I have a small VPS I purchased to test things, I may try to get this up and running. THANKS!

amonasro
12-15-2007, 02:32 PM
You can also download the firefox extension widget to monitor the donations.

http://ok4ronpaul.ashlux.com/wiki/index.php?title=Ron_Paul_Donations_Firefox_Extensi on

picoligance
12-15-2007, 04:41 PM
Hi, I've set up a mirror here: http://ronpaul.xnamatrix.com/

Got plenty of bandwidth to spare :) I used crontab instead of a script since I'm not sure if my host allows stray processes running around.

JacobLyles
12-15-2007, 04:46 PM
Don't forget http://paulcash.slact.net/ !

That's where I ended up going on 11/5 when http://www.ronpaulgraphs.com/ was overloaded... :)

Personally, I like paulcash better for monitoring one day's donations. I like ronpaulgraphs for looking at trends over time.

picoligance
12-15-2007, 04:58 PM
I just added the official widget and paulcash mirror too. Can't be too safe :)

Mirror here: http://ronpaul.xnamatrix.com/

paulforpaul
12-15-2007, 05:11 PM
I'm setting up a mirror right now.

Maybe we should let Dan know once we have a few mirrors ready so that he can do round-robin dns for the main domain name. I'll post an update when I have a url ready.

-Paul

shell
12-15-2007, 05:32 PM
g'day. i'm the guy that runs paulcash. I've tested it through and through, and I'm 95% confident that i will have absolutely no trouble coping with the traffic.

kill the banks
12-15-2007, 05:36 PM
g'day. i'm the guy that runs paulcash. I've tested it through and through, and I'm 95% confident that i will have absolutely no trouble coping with the traffic.
great job ... i use it all the time

kill the banks

paulforpaul
12-15-2007, 06:01 PM
Done...

I'm running an exact mirror ("wget -kp http://www.ronpaulgraphs.com/") at http://justkidding.voteneocon.com.

PM me if you have any creative ideas for the domain name voteneocon.com, which I am not using for anything other than this mirror at the moment.

Ronin
12-15-2007, 06:28 PM
Done...

I'm running an exact mirror ("wget -kp http://www.ronpaulgraphs.com/") at [.

Awesome! Maybe add the below flag to get just the *png images? That way you don't have to download each page? Not sure if that flag can be used with -kp though.

-A, --accept=LIST comma-separated list of accepted extensions.

paulforpaul
12-15-2007, 06:36 PM
At ~120 kb total for each time I update the mirror, it works out to be a grand total of 4.5 megabytes in bandwidth over 24 hours, at the rate I'm mirroring. I wrote Dan to make sure he's okay with the idea.

[update] nevermind, it works out to substantially more than that. I'll look into ways to reduce the load. Thanks for the idea Ronin.

chowda
12-15-2007, 07:20 PM
Hey guys... no need... I think everything is going to be fine.. I'm now on a dedicated server with unlimited access to bandwidth... I appreciate the thought, but mirroring is completely unnecessary. Like Leo I am 95%+ sure that there will be no interruption in service on the 16th.

chowda
12-15-2007, 07:21 PM
If you guys could watch my blog (see sig)... If I run into any problems, I will immediately post there...

I really appreciate it!

paulforpaul
12-15-2007, 07:25 PM
I'll be watching your blog, disabling my wget script, and doing a simple redirect from my URL to yours, just in case anyone tries to go there anyway.

Thanks for running ronpaulgraphs.com!

HelpRonPaulWin
12-15-2007, 07:27 PM
Sounds good, this is gonna be fun!

curiousobserver
12-15-2007, 07:43 PM
g'day. i'm the guy that runs paulcash. I've tested it through and through, and I'm 95% confident that i will have absolutely no trouble coping with the traffic.

Hey Shell,

can you change the Q4 total graphs' x-axis markers to be like 1M, 2M,...10M and so instead of putting it as 10000 (in $1000 of dollars).

I feel the 10'M' says it striaght without having to think about it twice.

Thanks. Hope my suggestion will be taken.

Ronin
12-15-2007, 08:05 PM
Thanks for running ronpaulgraphs.com!

+1000000

Benaiah
12-15-2007, 10:34 PM
blimp

picoligance
12-15-2007, 11:10 PM
Hey guys... no need... I think everything is going to be fine.. I'm now on a dedicated server with unlimited access to bandwidth... I appreciate the thought, but mirroring is completely unnecessary. Like Leo I am 95%+ sure that there will be no interruption in service on the 16th.

hehe, I think we just crashed your server for a few seconds there ;) Hopefully it's just a temporary spike rather than something you'll be struggling with throughout the day.

xexkxex
12-15-2007, 11:13 PM
Bump

Benaiah
12-16-2007, 12:04 AM
blimp