<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Eddie On Everything &#187; programming</title>
	<atom:link href="http://www.eddieoneverything.com/category/programming/feed" rel="self" type="application/rss+xml" />
	<link>http://www.eddieoneverything.com</link>
	<description>Tips &#38; tricks on things that interest me</description>
	<lastBuildDate>Wed, 11 Jan 2012 08:29:23 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.9.2</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>MySQL Performance Tip:  ON DUPLICATE KEY is faster than INSERT IGNORE, but failing with a duplicate key error is a lot faster</title>
		<link>http://www.eddieoneverything.com/programming/mysql-performance-tip-on-duplicate-key-is-faster-than-insert-ignore-but-failing-with-a-duplicate-key-error-is-a-lot-faster.php</link>
		<comments>http://www.eddieoneverything.com/programming/mysql-performance-tip-on-duplicate-key-is-faster-than-insert-ignore-but-failing-with-a-duplicate-key-error-is-a-lot-faster.php#comments</comments>
		<pubDate>Tue, 09 Mar 2010 15:02:52 +0000</pubDate>
		<dc:creator>eddie</dc:creator>
				<category><![CDATA[programming]]></category>
		<category><![CDATA[code]]></category>
		<category><![CDATA[database performance]]></category>
		<category><![CDATA[mysql]]></category>
		<category><![CDATA[mysql performance]]></category>
		<category><![CDATA[sql]]></category>
		<category><![CDATA[sql performance]]></category>
		<category><![CDATA[sql tuning]]></category>

		<guid isPermaLink="false">http://www.eddieoneverything.com/?p=1015</guid>
		<description><![CDATA[I&#8217;m working on merging a couple of pretty big MySQL databases.
Each database consists of just three tables, but each of those tables has at least 9 million rows.
Because I need to maintain and update a number of data associations, it was necessary to write a custom script to perform the merge.  (I chose to [...]]]></description>
			<content:encoded><![CDATA[<p>I&#8217;m working on merging a couple of pretty big MySQL databases.</p>
<p>Each database consists of just three tables, but each of those tables has at least 9 million rows.</p>
<p>Because I need to maintain and update a number of data associations, it was necessary to write a custom script to perform the merge.  (I chose to use Perl and DBI.)</p>
<p>At one point in my script, I need to do a bunch of &#8220;insert this row from database B into database A if it doesn&#8217;t exist in database A&#8221; type operations.  Doing a separate lookup for each record would cost roughly 9 million queries.  </p>
<p>At first, I just went ahead and did the insert, allowing the single insert to fail with a unique key error if the record already existed.  This caused a bunch of DBD errors to fill the screen, none of which really concerned me, because it didn&#8217;t have any bearing on the result. Still, all of those messages printed to STDERR could bury a more serious error, so I looked into handling it another way.</p>
<p><a href="http://bogdan.org.ua/2007/10/18/mysql-insert-if-not-exists-syntax.html" target="_blank">I found this post that recommends</a> using either INSERT IGNORE or ON DUPLICATE KEY to handle these queries.  After some investigation, I found that while ON DUPLICATE KEY is a lot faster than INSERT IGNORE, <em>just allowing the thing to fail is the fastest option</em>.</p>
<p>First, some simplified pseudo-code to give you an idea of what I&#8217;m doing:</p>
<pre name="code" class="vb">
FOR EACH RECORD IN A.TABLE1
   FOR EACH RECORD IN A.TABLE2
      IF THAT RECORD NOT EXISTS IN B.TABLE2,
      DO AN INSERT INTO B.TABLE2
</pre>
<p>I wanted to avoid doing a separate SQL query at step 3, as that query would have to run 9 million+ times.  Since I have a unique key set on the columns I&#8217;m interested in, any duplicate rows will not be inserted, thus maintaining my data integrity.</p>
<p>Here are the speed benchmarks I ran for my particular script with each of the three options I tried.  The numbers are in terms of iterations of my particular loop (which does a lot more than the pseudo-code above) so the numbers should only be considered relative to each other.</p>
<p><strong>Option 1:  Let the query fail with a Unique Key error.</strong></p>
<pre name="code" class="sql">
INSERT INTO A (COL1, COL2) VALUES (val1, val2);
</pre>
<p>Loop iterations per second: 24.5  (avg)</p>
<p><strong>Option 2:  INSERT &#8230; IGNORE</strong></p>
<pre name="code" class="sql">
INSERT IGNORE INTO A (COL1, COL2) VALUES (val1, val2);
</pre>
<p>Loop iterations per second: 13.3  (avg)</p>
<p><strong>Option 3:  INSERT &#8230; ON DUPLICATE KEY</strong></p>
<pre name="code" class="sql">
INSERT INTO A (COL1, COL2) VALUES (val1, val2) ON DUPLICATE KEY UPDATE id=id;
</pre>
<p>Loop iterations per second: 18.2  (avg)</p>
<p>As you can see, the fastest of the three options is letting the query fail, followed by ON DUPLICATE KEY as the second fastest.</p>
<p>I&#8217;ll leave it to the MySQL experts to explain why this is the case. I have some guesses, but I can&#8217;t say for sure.</p>
<p>But I do know one thing:  On my setup, for my purposes, letting those queries fail is the quickest option.</p>
<h2  class="related_post_title">Related Posts</h2><ul class="related_post"><li><a href="http://www.eddieoneverything.com/articles/managing-server-load-when-unzipping-many-large-files.php" title="Managing Server Load when unzipping many large files">Managing Server Load when unzipping many large files</a></li><li><a href="http://www.eddieoneverything.com/articles/how-to-scan-a-computer-for-open-ports.php" title="How to Scan a Computer for Open Ports">How to Scan a Computer for Open Ports</a></li><li><a href="http://www.eddieoneverything.com/articles/disabling-textarea-scrolling-in-flash-as3.php" title="Disabling TextArea Scrolling in Flash AS3">Disabling TextArea Scrolling in Flash AS3</a></li><li><a href="http://www.eddieoneverything.com/articles/the-zen-of-taco-bell-programming-using-unix-tools-to-prevent-reinventing-the-wheel.php" title="The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel">The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel</a></li><li><a href="http://www.eddieoneverything.com/articles/making-yahoo-sports-fantasy-pro-football-pickem-sortable.php" title="Making Yahoo! Sports Fantasy Pro Football Pickem Sortable">Making Yahoo! Sports Fantasy Pro Football Pickem Sortable</a></li><li><a href="http://www.eddieoneverything.com/articles/mod_rewrite-in-htaccess-to-force-www-in-url-not-working.php" title="mod_rewrite in .htaccess to Force www in URL Not Working ">mod_rewrite in .htaccess to Force www in URL Not Working </a></li><li><a href="http://www.eddieoneverything.com/programming/how-to-use-global-variables-from-another-file-in-a-perl-module-or-package.php" title="How To Use Global Variables From Another File In A Perl Module or Package">How To Use Global Variables From Another File In A Perl Module or Package</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.eddieoneverything.com/programming/mysql-performance-tip-on-duplicate-key-is-faster-than-insert-ignore-but-failing-with-a-duplicate-key-error-is-a-lot-faster.php/feed</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>How To Use Global Variables From Another File In A Perl Module or Package</title>
		<link>http://www.eddieoneverything.com/programming/how-to-use-global-variables-from-another-file-in-a-perl-module-or-package.php</link>
		<comments>http://www.eddieoneverything.com/programming/how-to-use-global-variables-from-another-file-in-a-perl-module-or-package.php#comments</comments>
		<pubDate>Sun, 07 Mar 2010 22:42:33 +0000</pubDate>
		<dc:creator>eddie</dc:creator>
				<category><![CDATA[programming]]></category>
		<category><![CDATA[code]]></category>
		<category><![CDATA[perl]]></category>
		<category><![CDATA[perl modules]]></category>
		<category><![CDATA[sharing variables]]></category>

		<guid isPermaLink="false">http://www.eddieoneverything.com/?p=991</guid>
		<description><![CDATA[I&#8217;ve been doing a lot of Perl development for my latest project, and I wanted to use a single config file for many scripts.
Using a single config file in a basic Perl script is pretty straightforward &#8211; you simply use a require statement to read the external file, and the variables contained within are automatically [...]]]></description>
			<content:encoded><![CDATA[<p>I&#8217;ve been doing a lot of Perl development for my latest project, and I wanted to use a single config file for many scripts.</p>
<p>Using a single config file in a basic Perl script is pretty straightforward &#8211; you simply use a require statement to read the external file, and the variables contained within are automatically imported.  (Though if you&#8217;re using strict, you&#8217;ll need to declare them using the &#8220;our&#8221; keyword.)</p>
<p>A problem arises when you want to use those global variables in a package file, though &#8211; because a Package has its own namespace, variables declared in your external config file are not imported into your module, even if you use the &#8220;our&#8221; keyword.</p>
<p>Fortunately, I was able to find a solution <a href="http://home.arcor.de/pangj/share_variables_between_perl_scripts.txt" target="_blank">here</a>, courtesy of Jeff Pang.</p>
<p>Here are a couple of examples.</p>
<p><strong>The simple way to share variables between Perl scripts using &#8220;require&#8221;</strong></p>
<p>File test.include.pl</p>
<pre name="code" class="perl">
#!/usr/bin/perl
$var = ".j.a.p.h.";
</pre>
<p>File test.pl</p>
<pre name="code" class="perl">
#!/usr/bin/perl
use strict;
require "test.include.pl";
our $var;
print "VAR is $var\n";
</pre>
<p>After creating the two files above and running test.pl, the output is:<br />
<code><br />
eddie@widmers:~/wrk/$ ./test.pl<br />
VAR is .j.a.p.h.<br />
</code></p>
<p><strong>The more advanced way of sharing variables with a Perl Package</strong><br />
This method involves declaring the config file as a package, and then explicitly exporting the variables you&#8217;re interested in.   Your other scripts and modules will then import those variables, making them available in that scope. </p>
<p>In this case, the config variable package will look something like this:</p>
<pre name="code" class="perl">
package SiteConfig;
require Exporter;

our @ISA = qw(Exporter);
our @EXPORT=qw($var $vare);

our $var = ".j.a.p.h.";

1;
</pre>
<p>While the package you&#8217;re looking to use the variables in will look something like this:</p>
<pre name="code" class="perl">
package TestTest;
use TestConfig;
print "Var is $var\n";

sub new
{
   my $package = shift;
   print "VAR is $var\n";
  return bless({}, $package);
}
1;
</pre>
<p>Of course, your particular needs will vary, so you&#8217;ll need to use your noodle to modify the above as necessary.</p>
<h2  class="related_post_title">Related Posts</h2><ul class="related_post"><li><a href="http://www.eddieoneverything.com/linux/argument-list-too-long-from-rm-command-a-perl-script-to-get-around-the-problem.php" title="&#8220;Argument list too long&#8221; from rm command.   A Perl Script to Get Around the Problem">&#8220;Argument list too long&#8221; from rm command.   A Perl Script to Get Around the Problem</a></li><li><a href="http://www.eddieoneverything.com/articles/managing-server-load-when-unzipping-many-large-files.php" title="Managing Server Load when unzipping many large files">Managing Server Load when unzipping many large files</a></li><li><a href="http://www.eddieoneverything.com/articles/how-to-scan-a-computer-for-open-ports.php" title="How to Scan a Computer for Open Ports">How to Scan a Computer for Open Ports</a></li><li><a href="http://www.eddieoneverything.com/articles/the-zen-of-taco-bell-programming-using-unix-tools-to-prevent-reinventing-the-wheel.php" title="The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel">The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel</a></li><li><a href="http://www.eddieoneverything.com/programming/getting-phps-print_r-function-to-return-a-string.php" title="Getting PHP&#8217;s print_r Function To Return A String">Getting PHP&#8217;s print_r Function To Return A String</a></li><li><a href="http://www.eddieoneverything.com/programming/calibre-recipe-for-minneapolis-star-tribune-startribune-com.php" title="Calibre Recipe for Minneapolis Star Tribune (StarTribune.com)">Calibre Recipe for Minneapolis Star Tribune (StarTribune.com)</a></li><li><a href="http://www.eddieoneverything.com/linux/rotating-backup-directories-using-cp-al-hardlinks-to-save-disk-space.php" title="Rotating Backup Directories using cp -al (hardlinks) to Save Disk Space">Rotating Backup Directories using cp -al (hardlinks) to Save Disk Space</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.eddieoneverything.com/programming/how-to-use-global-variables-from-another-file-in-a-perl-module-or-package.php/feed</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Getting PHP&#8217;s print_r Function To Return A String</title>
		<link>http://www.eddieoneverything.com/programming/getting-phps-print_r-function-to-return-a-string.php</link>
		<comments>http://www.eddieoneverything.com/programming/getting-phps-print_r-function-to-return-a-string.php#comments</comments>
		<pubDate>Thu, 25 Feb 2010 22:41:42 +0000</pubDate>
		<dc:creator>eddie</dc:creator>
				<category><![CDATA[programming]]></category>
		<category><![CDATA[code]]></category>
		<category><![CDATA[debugging]]></category>
		<category><![CDATA[PHP]]></category>
		<category><![CDATA[print_r]]></category>

		<guid isPermaLink="false">http://www.eddieoneverything.com/?p=895</guid>
		<description><![CDATA[PHP&#8217;s print_r function is invaluable.  It prints a human-readable string representation of a variable.
It&#8217;s one of the most useful debugging features I&#8217;ve seen in any language.
I like it so much that I&#8217;ve actually written print_r mimic functions for other languages.
By default, print_r literally prints a variable to the screen.  Sometimes it&#8217;s useful to [...]]]></description>
			<content:encoded><![CDATA[<p>PHP&#8217;s print_r function is invaluable.  It prints a human-readable string representation of a variable.</p>
<p>It&#8217;s one of the most useful debugging features I&#8217;ve seen in any language.</p>
<p>I like it so much that I&#8217;ve actually written print_r mimic functions for other languages.</p>
<p>By default, print_r literally <i>prints</i> a variable to the screen.  Sometimes it&#8217;s useful to have that info not printed though.  For instance, you might want to include the print_r result in an email, or an error log, or in a special debugging frame.</p>
<p>Fortunately, print_r has this functionality built in &#8211; the function accepts an optional second argument that specifies whether you want the data printed or returned as a string.</p>
<p>To have print_r return the data as a string, include the second argument &#8220;true&#8221;:</p>
<pre name="code" class="php">
$message = print_r($myObject,true);
</pre>
<p>If you&#8217;re doing anything programmatic with the output, you may want to consider using var_export instead.  This function is largely identical to print_r, except that its output is machine parsable rather than human readable.  The usage is identical to print_r.</p>
<pre name="code" class="php">
$message = var_export($myObject,true);
</pre>
<p>All without using ob_start.  Nice!</p>
<h2  class="related_post_title">Related Posts</h2><ul class="related_post"><li><a href="http://www.eddieoneverything.com/programming/php-programming-us-state-list-functions-state-abbreviation-to-state-name-function.php" title="PHP Programming: U.S. State list functions / State abbreviation to State name function">PHP Programming: U.S. State list functions / State abbreviation to State name function</a></li><li><a href="http://www.eddieoneverything.com/articles/validating-an-email-address-with-phps-filter_var-isnt-perfect.php" title="Validating an Email Address with PHP&rsquo;s filter_var isn&rsquo;t perfect">Validating an Email Address with PHP&rsquo;s filter_var isn&rsquo;t perfect</a></li><li><a href="http://www.eddieoneverything.com/articles/how-to-scan-a-computer-for-open-ports.php" title="How to Scan a Computer for Open Ports">How to Scan a Computer for Open Ports</a></li><li><a href="http://www.eddieoneverything.com/articles/the-zen-of-taco-bell-programming-using-unix-tools-to-prevent-reinventing-the-wheel.php" title="The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel">The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel</a></li><li><a href="http://www.eddieoneverything.com/articles/how-to-make-phpbb3-forum-links-automatically-relnofollow.php" title="How to Make phpbb3 Forum Links Automatically rel=&#8221;nofollow&#8221;">How to Make phpbb3 Forum Links Automatically rel=&#8221;nofollow&#8221;</a></li><li><a href="http://www.eddieoneverything.com/programming/how-to-use-global-variables-from-another-file-in-a-perl-module-or-package.php" title="How To Use Global Variables From Another File In A Perl Module or Package">How To Use Global Variables From Another File In A Perl Module or Package</a></li><li><a href="http://www.eddieoneverything.com/articles/optimizing-the-google-syntax-highlighter-wordpress-plugin-even-further.php" title="Optimizing the Google Syntax Highlighter Wordpress Plugin Even Further">Optimizing the Google Syntax Highlighter Wordpress Plugin Even Further</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.eddieoneverything.com/programming/getting-phps-print_r-function-to-return-a-string.php/feed</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
		<item>
		<title>How To Set a Class Name using Javascript in IE and Other Browsers</title>
		<link>http://www.eddieoneverything.com/programming/how-to-set-a-class-name-using-javascript-in-ie-and-other-browsers.php</link>
		<comments>http://www.eddieoneverything.com/programming/how-to-set-a-class-name-using-javascript-in-ie-and-other-browsers.php#comments</comments>
		<pubDate>Thu, 25 Feb 2010 20:35:48 +0000</pubDate>
		<dc:creator>eddie</dc:creator>
				<category><![CDATA[programming]]></category>
		<category><![CDATA[code]]></category>
		<category><![CDATA[HTML]]></category>
		<category><![CDATA[IE]]></category>
		<category><![CDATA[javascript]]></category>

		<guid isPermaLink="false">http://www.eddieoneverything.com/?p=890</guid>
		<description><![CDATA[I was working a client project today had to use a little javascript.
As anyone who&#8217;s worked with Javascript knows, Microsoft&#8217;s Internet Explorer can be a real pain the butt.
Things that work in Chrome, Firefox, and Safari don&#8217;t always work in IE &#8211; so you always have to come up with workarounds to make IE, well, [...]]]></description>
			<content:encoded><![CDATA[<p>I was working a client project today had to use a little javascript.</p>
<p>As anyone who&#8217;s worked with Javascript knows, Microsoft&#8217;s Internet Explorer can be a real pain the butt.</p>
<p>Things that work in Chrome, Firefox, and Safari don&#8217;t always work in IE &#8211; so you always have to come up with workarounds to make IE, well, <em>work</em>.</p>
<p>One of those things that has to be done differently is setting a class name.  </p>
<p>For whatever reason, IE does not support using &#8220;setAttribute&#8221; to set class names.  Instead, in IE, you have to use <em>className = newClass</em>.</p>
<p>Here is a little bit of code that should do the trick:</p>
<pre name="code" class="javascript">
if (navigator.appName == "Microsoft Internet Explorer") {
     document.getElementById("id").className="new_class";
}else{
     document.getElementById("id").setAttribute("class", "new_class");
}
</pre>
<h2  class="related_post_title">Related Posts</h2><ul class="related_post"><li><a href="http://www.eddieoneverything.com/articles/how-to-set-the-window-size-and-position-of-a-chrome-application-shortcut.php" title="How to Set the Window Size and Position of a Chrome Application Shortcut">How to Set the Window Size and Position of a Chrome Application Shortcut</a></li><li><a href="http://www.eddieoneverything.com/articles/how-to-block-a-quote.php" title="How to Block a Quote">How to Block a Quote</a></li><li><a href="http://www.eddieoneverything.com/internet/how-to-reveal-stored-passwords-in-firefox-chrome-safari-and-ie.php" title="How To Reveal Stored Passwords in Firefox, Chrome, Safari, and IE">How To Reveal Stored Passwords in Firefox, Chrome, Safari, and IE</a></li><li><a href="http://www.eddieoneverything.com/programming/calculate-the-days-since-a-given-date-javascript.php" title="Calculate the Days Since a Given Date (Javascript)">Calculate the Days Since a Given Date (Javascript)</a></li><li><a href="http://www.eddieoneverything.com/articles/managing-server-load-when-unzipping-many-large-files.php" title="Managing Server Load when unzipping many large files">Managing Server Load when unzipping many large files</a></li><li><a href="http://www.eddieoneverything.com/articles/how-to-scan-a-computer-for-open-ports.php" title="How to Scan a Computer for Open Ports">How to Scan a Computer for Open Ports</a></li><li><a href="http://www.eddieoneverything.com/articles/disabling-textarea-scrolling-in-flash-as3.php" title="Disabling TextArea Scrolling in Flash AS3">Disabling TextArea Scrolling in Flash AS3</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.eddieoneverything.com/programming/how-to-set-a-class-name-using-javascript-in-ie-and-other-browsers.php/feed</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Customizing the Calibre Associated Press (AP) Recipe</title>
		<link>http://www.eddieoneverything.com/programming/customizing-the-calibre-associated-press-ap-recipe.php</link>
		<comments>http://www.eddieoneverything.com/programming/customizing-the-calibre-associated-press-ap-recipe.php#comments</comments>
		<pubDate>Mon, 22 Feb 2010 16:48:18 +0000</pubDate>
		<dc:creator>eddie</dc:creator>
				<category><![CDATA[programming]]></category>
		<category><![CDATA[amazon kindle]]></category>
		<category><![CDATA[AP]]></category>
		<category><![CDATA[associated press]]></category>
		<category><![CDATA[calibre]]></category>
		<category><![CDATA[calibre recipes]]></category>
		<category><![CDATA[code]]></category>
		<category><![CDATA[kindle]]></category>
		<category><![CDATA[python]]></category>

		<guid isPermaLink="false">http://www.eddieoneverything.com/?p=761</guid>
		<description><![CDATA[I&#8217;ve been using Calibre to manage my eBook library and download daily news to my Kindle.   It&#8217;s great.  If you have a Kindle or other eBook reader, I recommend that you check it out.
Calibre comes stocked with a number of &#8220;recipes&#8221; for popular news sites.  One of those is the Associated Press (AP).  Since [...]]]></description>
			<content:encoded><![CDATA[<p>I&#8217;ve been using Calibre to manage my eBook library and download daily news to my Kindle.   It&#8217;s great.  If you have a Kindle or other eBook reader, I recommend that you check it out.</p>
<p>Calibre comes stocked with a number of &#8220;recipes&#8221; for popular news sites.  One of those is the Associated Press (AP).  Since the AP creates a good deal of the news in many popular dailies,  I&#8217;ve found the AP recipe most useful.</p>
<p>I noticed today, though, that the AP recipe that comes with Calibre is incomplete!  It omits at least three sections, Sports, Business, and Entertainment.  I don&#8217;t know if the creator, Kovid Goyal,  just doesn&#8217;t like these sections, or if they weren&#8217;t available when he wrote the original recipe.</p>
<p>(When I wrote my <a href="http://www.eddieoneverything.com/programming/calibre-recipe-for-minneapolis-star-tribune-startribune-com.php" target="_blank">custom recipe for the Minneapolis Star-Tribune</a>, I omitted the sections that weren&#8217;t of interest to me.)</p>
<p>In any event, I&#8217;ve customized his recipe script to include the omitted sections.  It was a very easy customization &#8211; I only had to add three lines to the original python script.  The only tricky part was picking a local &#8220;news source&#8221; for the AP&#8217;s RSS feed &#8211; otherwise, the AP site serves up a random source, which can lead to problems with processing the page.</p>
<pre name="code" class="python">
import re
from calibre.web.feeds.news import BasicNewsRecipe

class AssociatedPress(BasicNewsRecipe):

    title = u'Associated Press'
    description = 'Global news'
    __author__ = 'Kovid Goyal'
    use_embedded_content   = False
    language = 'en'

    max_articles_per_feed = 15
    html2lrf_options = ['--force-page-break-before-tag="chapter"']

    preprocess_regexps = [ (re.compile(i[0], re.IGNORECASE | re.DOTALL), i[1]) for i in
[
        (r'<HEAD>.*?</HEAD>' , lambda match : '<HEAD></HEAD>'),
        (r'<body class="apple-rss-no-unread-mode" onLoad="setup(null)">.*?<!-- start Entries -->', lambda match : '<body>'),
        (r'<!-- end apple-rss-content-area -->.*?</body>', lambda match : '</body>'),
        (r'<script.*?>.*?</script>', lambda match : ''),
        (r'<body.*?>.*?<span class="headline">', lambda match : '<body><span class="headline"><chapter>'),
        (r'
<tr>
<td>
<div class="body">.*?
<p class="ap-story-p">', lambda match : '
<p class="ap-story-p">'),
        (r'
<p class="ap-story-p">', lambda match : '

'),
        (r'Learn more about our <a href="http://apdigitalnews.com/privacy.html">Privacy Policy</a>.*?</body>', lambda match : '</body>'),
    ]
    ]   

    feeds = [ ('AP Headlines', 'http://hosted.ap.org/lineups/TOPHEADS-rss_2.0.xml?SITE=ORAST&#038;SECTION=HOME'),
                  ('AP US News', 'http://hosted.ap.org/lineups/USHEADS-rss_2.0.xml?SITE=CAVIC&#038;SECTION=HOME'),
                   ('AP World News', 'http://hosted.ap.org/lineups/WORLDHEADS-rss_2.0.xml?SITE=SCAND&#038;SECTION=HOME'),
                   ('AP Political News', 'http://hosted.ap.org/lineups/POLITICSHEADS-rss_2.0.xml?SITE=ORMED&#038;SECTION=HOME'),
                   ('AP Business News', 'http://hosted.ap.org/lineups/BUSINESSHEADS-rss_2.0.xml?SITE=RANDOM&#038;SECTION=HOME'),
                   ('AP Technology News', 'http://hosted.ap.org/lineups/TECHHEADS-rss_2.0.xml?SITE=CTNHR&#038;SECTION=HOME'),
                   ('AP Sports News', 'http://hosted.ap.org/lineups/SPORTSHEADS-rss_2.0.xml?SITE=CAVIC&#038;SECTION=HOME'),
                   ('AP Business News', 'http://hosted.ap.org/lineups/BUSINESSHEADS-rss_2.0.xml?SITE=CAVIC&#038;SECTION=HOME'),
                   ('AP Entertainment News', 'http://hosted.ap.org/lineups/ENTERTAINMENT-rss_2.0.xml?SITE=CAVIC&#038;SECTION=HOME'),
                   ('AP Science News', 'http://hosted.ap.org/lineups/SCIENCEHEADS-rss_2.0.xml?SITE=OHCIN&#038;SECTION=HOME'),
                   ('AP Strange News', 'http://hosted.ap.org/lineups/STRANGEHEADS-rss_2.0.xml?SITE=WCNC&#038;SECTION=HOME'),
        ]
</pre>
<h2  class="related_post_title">Related Posts</h2><ul class="related_post"><li><a href="http://www.eddieoneverything.com/programming/calibre-recipe-for-minneapolis-star-tribune-startribune-com.php" title="Calibre Recipe for Minneapolis Star Tribune (StarTribune.com)">Calibre Recipe for Minneapolis Star Tribune (StarTribune.com)</a></li><li><a href="http://www.eddieoneverything.com/articles/getting-a-list-of-all-books-in-your-calibre-library.php" title="Getting a List of All Books in Your Calibre Library">Getting a List of All Books in Your Calibre Library</a></li><li><a href="http://www.eddieoneverything.com/articles/my-kindles-screen-is-broken-but-amazon-is-replacing-it-for-free.php" title="My Kindle&rsquo;s Screen is Broken! But Amazon is Replacing It &ndash; For Free">My Kindle&rsquo;s Screen is Broken! But Amazon is Replacing It &ndash; For Free</a></li><li><a href="http://www.eddieoneverything.com/articles/changing-the-margin-size-on-the-kindle.php" title="Changing the Margin Size on the Kindle">Changing the Margin Size on the Kindle</a></li><li><a href="http://www.eddieoneverything.com/articles/gwbs-decision-points-kindle-ebook-more-expensive-than-actual-printed-copy.php" title="GWB&#8217;s &#8220;Decision Points&#8221; Kindle eBook more expensive than actual printed copy">GWB&#8217;s &#8220;Decision Points&#8221; Kindle eBook more expensive than actual printed copy</a></li><li><a href="http://www.eddieoneverything.com/articles/how-to-buy-usuk-restricted-kindle-ebook-books-on-amazon.php" title="How to Buy US/UK-Only Restricted eBooks on Amazon Kindle">How to Buy US/UK-Only Restricted eBooks on Amazon Kindle</a></li><li><a href="http://www.eddieoneverything.com/articles/how-to-tell-if-someone-is-lying-about-their-ipad-experience.php" title="How To Tell If Someone Is Lying About Their iPad Experience">How To Tell If Someone Is Lying About Their iPad Experience</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.eddieoneverything.com/programming/customizing-the-calibre-associated-press-ap-recipe.php/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>What&#8217;s the difference between the Strong and the Bold tag in HTML?  &lt;Strong&gt; vs. &lt;B&gt;?</title>
		<link>http://www.eddieoneverything.com/programming/whats-the-difference-between-the-strong-and-the-bold-tag-in-html.php</link>
		<comments>http://www.eddieoneverything.com/programming/whats-the-difference-between-the-strong-and-the-bold-tag-in-html.php#comments</comments>
		<pubDate>Sun, 21 Feb 2010 00:16:44 +0000</pubDate>
		<dc:creator>eddie</dc:creator>
				<category><![CDATA[programming]]></category>
		<category><![CDATA[HTML]]></category>
		<category><![CDATA[HTML b]]></category>
		<category><![CDATA[HTML bold]]></category>
		<category><![CDATA[HTML conventions]]></category>
		<category><![CDATA[HTML em]]></category>
		<category><![CDATA[HTML emphasis]]></category>
		<category><![CDATA[HTML i]]></category>
		<category><![CDATA[HTML italics]]></category>
		<category><![CDATA[HTML presentation]]></category>
		<category><![CDATA[HTML semantics]]></category>
		<category><![CDATA[HTML strong]]></category>
		<category><![CDATA[HTML structure]]></category>
		<category><![CDATA[WWW]]></category>
		<category><![CDATA[WWW publishing]]></category>

		<guid isPermaLink="false">http://www.eddieoneverything.com/?p=710</guid>
		<description><![CDATA[A few versions back, Wordpress&#8217; WYSIWYG editor started using the &#60;strong&#62; tag instead of the &#60;b&#62; tag.
And it wasn&#8217;t just Wordpress &#8211; a lot of software quietly made the shift was well.  Dreamweaver, for one.
Why the change?  What&#8217;s the difference between the two HTML tags?
As with most arguments of a religious nature, there&#8217;s both a [...]]]></description>
			<content:encoded><![CDATA[<p>A few versions back, Wordpress&#8217; WYSIWYG editor started using the &lt;strong&gt; tag instead of the &lt;b&gt; tag.</p>
<p>And it wasn&#8217;t just Wordpress &#8211; a lot of software quietly made the shift was well.  Dreamweaver, for one.</p>
<p>Why the change?  What&#8217;s the difference between the two HTML tags?</p>
<p>As with most arguments of a religious nature, there&#8217;s both a short and a long answer to those questions.</p>
<p><strong>Here&#8217;s the short answer: </strong> Functionally, there&#8217;s no difference whatsoever.  Browsers render the tag exactly the same way.</p>
<p><strong>And now, the long answer:</strong> But that short answer isn&#8217;t quite correct.  While there isn&#8217;t a functional difference, there <em>is</em> a semantic difference between &#8220;bold&#8221; and &#8220;strong.&#8221;   And it goes like this:</p>
<ul>
<li>In a piece of writing,  &#8221;bold&#8221; is a strictly <em>presentational</em> element &#8211; that is, when you say &#8220;bold,&#8221; you are instructing whatever is rendering the text to increase a font-weight.</li>
<li>&#8220;Strong,&#8221; on the other hand, is a <em>structural</em> element.  Strong says nothing about how text should be rendered &#8211; instead, it says &#8220;in the context of this written piece, this passage should be read/spoken with strong emphasis.&#8221;</li>
</ul>
<p>It may help to think of the situation in terms of a spoken piece, rather than a written piece.  In a spoken piece, if you want a speaker to emphasize a particular passage, you use the term &#8220;strong,&#8221; not &#8220;bold.&#8221;  You say, &#8220;This line should be spoken strongly!  With strong emphasis!&#8221;  Not &#8220;Say this line in bold.&#8221;</p>
<p>What we&#8217;re really talking about here is the <a href="http://en.wikipedia.org/wiki/Separation_of_presentation_and_content" target="_blank">separation of presentation and content</a>.   While the topic may seem trivial to many, to others it conjures up feelings as strong (or as bold) as one&#8217;s own religious convictions.  And for that reason, I&#8217;m going to steer clear.</p>
<p>Keep in mind that these distinctions also apply to the difference between the italicize tag, &#8220;&lt;i&gt;&#8221;, and the emphasis tag, or &#8220;&lt;em&gt;&#8221;.</p>
<p><strong>So which one should I use, &lt;strong&gt; or &lt;b&gt;?</strong> Technically, &#8220;&lt;strong&gt;&#8221; is correct when you are trying to emphasize a particular word or passage.  <a href="http://www.w3.org/" target="_blank">The World Wide Web Consortium (w3c)</a> recommends using &lt;strong&gt;, as it separates the presentation of the content from the content itself.</p>
<p>I suppose one could argue that using &#8220;&lt;b&gt;&#8221; rather than &#8220;&lt;strong&gt;&#8221; is correct when you are bolding something solely for presentation&#8217;s sake &#8211; like a section heading or visual marker of some sort. The W3C would probably disagree, saying that you should instead use a header tag (e.g. &lt;h3&gt;).  The W3C hates &lt;b&gt;, and loves well-structured content.</p>
<p>Still, you won&#8217;t find many downsides to using &#8220;&lt;b&gt;&#8221;.  So unless you&#8217;re in a large organization with guidelines on this topic, I&#8217;d say to go ahead and use whichever you prefer.</p>
<p>When I&#8217;m coding HTML by hand, I don&#8217;t like the extra keystrokes it takes to type those extra 5 letters, so I use a simple &lt;b&gt; tag.  When I use Wordpress&#8217; WYSIWYG editor, it uses the &#8220;&lt;strong&gt;&#8221; tag automatically, and I don&#8217;t mind one bit.   On this site, like most others, there is no functional difference between &#8220;strong&#8221; and &#8220;bold.&#8221;  So for me, (and probably for you, too,) it really doesn&#8217;t matter.</p>
<h2  class="related_post_title">Related Posts</h2><ul class="related_post"><li><a href="http://www.eddieoneverything.com/articles/how-to-set-the-window-size-and-position-of-a-chrome-application-shortcut.php" title="How to Set the Window Size and Position of a Chrome Application Shortcut">How to Set the Window Size and Position of a Chrome Application Shortcut</a></li><li><a href="http://www.eddieoneverything.com/articles/how-to-block-a-quote.php" title="How to Block a Quote">How to Block a Quote</a></li><li><a href="http://www.eddieoneverything.com/programming/how-to-set-a-class-name-using-javascript-in-ie-and-other-browsers.php" title="How To Set a Class Name using Javascript in IE and Other Browsers">How To Set a Class Name using Javascript in IE and Other Browsers</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.eddieoneverything.com/programming/whats-the-difference-between-the-strong-and-the-bold-tag-in-html.php/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Calibre Recipe for Minneapolis Star Tribune (StarTribune.com)</title>
		<link>http://www.eddieoneverything.com/programming/calibre-recipe-for-minneapolis-star-tribune-startribune-com.php</link>
		<comments>http://www.eddieoneverything.com/programming/calibre-recipe-for-minneapolis-star-tribune-startribune-com.php#comments</comments>
		<pubDate>Mon, 15 Feb 2010 22:55:04 +0000</pubDate>
		<dc:creator>eddie</dc:creator>
				<category><![CDATA[programming]]></category>
		<category><![CDATA[amazon kindle]]></category>
		<category><![CDATA[calibre]]></category>
		<category><![CDATA[code]]></category>
		<category><![CDATA[kindle]]></category>
		<category><![CDATA[python]]></category>

		<guid isPermaLink="false">http://www.eddieoneverything.com/?p=628</guid>
		<description><![CDATA[I love my Kindle.  And I love the Calibre eBook management software.  If you haven&#8217;t had a chance to check out Calibre, you&#8217;re missing out &#8211; it&#8217;s kinda like an iTunes for your eBook collection.  In addition to storing, converting, updating, and categorizing your eBook collection, the software will also download news [...]]]></description>
			<content:encoded><![CDATA[<p>I love my Kindle.  And I love the <a href="http://calibre-ebook.com/" target="blank">Calibre eBook management software</a>.  If you haven&#8217;t had a chance to check out Calibre, you&#8217;re missing out &#8211; it&#8217;s kinda like an iTunes for your eBook collection.  In addition to storing, converting, updating, and categorizing your eBook collection, the software will also download news from any website, and will format that news for proper viewing on your Kindle or other eBook reader. </p>
<p>Calibre comes stocked with &#8220;recipes&#8221; for downloading news from hundreds of websites.  Unfortunately, the Minneapolis Star Tribune is not one of them. So I wrote my own.</p>
<p>Here is my very simple recipe for the Star Tribune.  It omits many of the newspaper sections that I am not interested in, so you may want to edit my list of sources using <a href="http://www.startribune.com/help/12374506.html">this list</a> of RSS feeds as your guide.  It&#8217;s written in Python, so spacing is important.</p>
<p><strong>Calibre recipe for the Minneapolis Star Tribune</strong></p>
<pre name="code" class="python">
class AdvancedUserRecipe1266267637(BasicNewsRecipe):
    title          = u'Star Tribune'
    oldest_article = 7
    max_articles_per_feed = 100

    feeds          = [(u'Main', u'http://www.startribune.com/rss/?sf=1&#038;s=/'), (u'Local', u'http://www.startribune.com/local/index.rss2'), (u'Minneapolis', u'http://www.startribune.com/local/minneapolis/index.rss2'), (u'St. Paul', u'http://www.startribune.com/local/stpaul/index.rss2'), (u'North Metro', u'http://www.startribune.com/local/north/index.rss2'), (u'East Metro', u'http://www.startribune.com/local/east/index.rss2'), (u'South Metro', u'http://www.startribune.com/local/south/index.rss2'), (u'West Metro', u'http://www.startribune.com/local/west/index.rss2'), (u'Business', u'http://www.startribune.com/business/index.rss2'), (u'Science &#038; Technology', u'http://www.startribune.com/science/index.rss2'), (u'Sports', u'http://www.startribune.com/sports/index.rss2'), (u'Lifestyle', u'http://www.startribune.com/lifestyle/index.rss2'), (u'Travel', u'http://www.startribune.com/lifestyle/travel/index.rss2'), (u'Books', u'http://www.startribune.com/entertainment/books/index.rss2'), (u'Entertainment', u'http://www.startribune.com/entertainment/index.rss2'), (u'Commentary', u'http://www.startribune.com/opinion/commentary/index.rss2'), (u'Editorials', u'http://www.startribune.com/opinion/editorials/index.rss2')]

    def print_version(self, url):
        parts=url.split('/')
        id = parts[4].split('.')
        if not id[0].isdigit():
            id = parts[5].split('.')

        return url.replace(url, 'http://www.startribune.com/templates/Print_This_Story?sid=' + id[0])
</pre>
<p>I&#8217;ve also <a href="http://bugs.calibre-ebook.com/attachment/wiki/UserRecipes/StarTribune.py">contributed</a> this code to the wiki repository for <a href="http://bugs.calibre-ebook.com/wiki/UserRecipes">user-created recipes</a>.</p>
<p>There is a small bug &#8211; I&#8217;ve noticed that some stories in the Sports section use some sort of Redirect to steer users to a blog entry.  Because of the way the Star Tribune people are doing this redirect, the recipe doesn&#8217;t work on those stories.  It&#8217;s only a very few stories, though, and I&#8217;m fine with it, so I&#8217;m not going to spend time tweaking the recipe.</p>
<h2  class="related_post_title">Related Posts</h2><ul class="related_post"><li><a href="http://www.eddieoneverything.com/programming/customizing-the-calibre-associated-press-ap-recipe.php" title="Customizing the Calibre Associated Press (AP) Recipe">Customizing the Calibre Associated Press (AP) Recipe</a></li><li><a href="http://www.eddieoneverything.com/articles/getting-a-list-of-all-books-in-your-calibre-library.php" title="Getting a List of All Books in Your Calibre Library">Getting a List of All Books in Your Calibre Library</a></li><li><a href="http://www.eddieoneverything.com/articles/my-kindles-screen-is-broken-but-amazon-is-replacing-it-for-free.php" title="My Kindle&rsquo;s Screen is Broken! But Amazon is Replacing It &ndash; For Free">My Kindle&rsquo;s Screen is Broken! But Amazon is Replacing It &ndash; For Free</a></li><li><a href="http://www.eddieoneverything.com/articles/changing-the-margin-size-on-the-kindle.php" title="Changing the Margin Size on the Kindle">Changing the Margin Size on the Kindle</a></li><li><a href="http://www.eddieoneverything.com/articles/how-to-scan-a-computer-for-open-ports.php" title="How to Scan a Computer for Open Ports">How to Scan a Computer for Open Ports</a></li><li><a href="http://www.eddieoneverything.com/articles/gwbs-decision-points-kindle-ebook-more-expensive-than-actual-printed-copy.php" title="GWB&#8217;s &#8220;Decision Points&#8221; Kindle eBook more expensive than actual printed copy">GWB&#8217;s &#8220;Decision Points&#8221; Kindle eBook more expensive than actual printed copy</a></li><li><a href="http://www.eddieoneverything.com/articles/the-zen-of-taco-bell-programming-using-unix-tools-to-prevent-reinventing-the-wheel.php" title="The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel">The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.eddieoneverything.com/programming/calibre-recipe-for-minneapolis-star-tribune-startribune-com.php/feed</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
		<item>
		<title>Rotating Backup Directories using cp -al (hardlinks) to Save Disk Space</title>
		<link>http://www.eddieoneverything.com/linux/rotating-backup-directories-using-cp-al-hardlinks-to-save-disk-space.php</link>
		<comments>http://www.eddieoneverything.com/linux/rotating-backup-directories-using-cp-al-hardlinks-to-save-disk-space.php#comments</comments>
		<pubDate>Sun, 15 Nov 2009 20:13:16 +0000</pubDate>
		<dc:creator>eddie</dc:creator>
				<category><![CDATA[linux]]></category>
		<category><![CDATA[perl]]></category>
		<category><![CDATA[programming]]></category>
		<category><![CDATA[backup]]></category>
		<category><![CDATA[code]]></category>
		<category><![CDATA[scripts]]></category>
		<category><![CDATA[system administration]]></category>
		<category><![CDATA[ubuntu]]></category>
		<category><![CDATA[unix]]></category>

		<guid isPermaLink="false">http://www.eddieoneverything.com/?p=356</guid>
		<description><![CDATA[The copy command &#8220;cp -al&#8221; found on all versions of Unix/Linux creates what&#8217;s called a &#8220;hard link&#8221; to a file. The nice thing about this command is that it doesn&#8217;t create an actual copy of the file on disk &#8211; instead, it creates a &#8220;link&#8221; or pointer to the file data on the disk.  [...]]]></description>
			<content:encoded><![CDATA[<p>The copy command &#8220;cp -al&#8221; found on all versions of Unix/Linux creates what&#8217;s called a &#8220;hard link&#8221; to a file. The nice thing about this command is that it doesn&#8217;t create an actual copy of the file on disk &#8211; instead, it creates a &#8220;link&#8221; or pointer to the file data on the disk.  Basically a &#8220;snapshot&#8221; of that directory in-time.   The net result is that you can have 10 &#8220;copies&#8217; of a 10G file that only take up a total of 10G. </p>
<p>This nifty behavior makes cp -al, when combined with rsync,  ideal for backup systems.  One can use the cp -al command to take a &#8220;snapshot&#8221; of a given directory tree at a given time, at the expense of very little additional disk space.  I use this script in concert with my <a href="http://www.eddieoneverything.com/linux/using-rsync-ssh-to-backup-files-between-linux-machines-perl.php">rsync_backup.pl</a> script to keep 21 days of &#8220;snapshot&#8221; backups of each of my machines.</p>
<pre name="code" class="perl">

#!/usr/bin/perl
use POSIX;

# Rotates backup directories w/ cp -al (hardlinks)
# Deletes directories older than $KEEP_DAYS
# Runs each night ahead of backup process
# (c) 2009 eddie@eddieoneverything.com

$KEEP_DAYS=21;
$LOGFILE = "/var/log/rotate_backups";

@BACKUP_DIRS=(
   '/mnt/backup/hansel',
   '/mnt/backup/tiger',
   '/mnt/backup/june'
);

$ts = get_timestamp();

open hLOG, ">>$LOGFILE";
print hLOG "=" x 80, "\n";
print hLOG "Run START at " . `date` . "\n";
print hLOG "=" x 80, "\n";

## Do the rotation
print hLOG "Do today's rotation\n";
foreach $dir ( @BACKUP_DIRS ){
   print hLOG "\t" , `date`;
   $newfn = $dir ."_" .  $ts;
   $cmd = "cp -al $dir $newfn";
   print hLOG "Execute Command: $cmd\n";
   `$cmd`;
}

## Delete old directories
print hLOG "Delete Old Directories\n";
foreach $dir ( @BACKUP_DIRS ){
   $dir=~m/^(.+)\/(.+?)$/;
   $base= $1;
   $stub=$2;
   #print "dir is $dir\nBASE: $base\nSTUB:$stub\n";

   opendir hDIR, "$base" or die "can't open directory $base";
   @dirlist=grep { /^$stub\_/ &#038;&#038; !/^$stub$/  } readdir(hDIR);
   closedir hDIR;

   foreach $d (@dirlist){
      print hLOG "\t" , `date`;
      $d=~/$stub\_([0-9]+)_([0-9]+)_([0-9]+)_.+$/;
      $year= $1;
      $month = $2;
      $day = $3;

      if (dirIsOlder($year, $month, $day)){
         #print "$d\n";
         $remove_dir= $base . '/' . $d;
         $cmd = "rm -Rf \"$remove_dir\"";
         print hLOG "Execute command: $cmd\n";
         `$cmd`;
      }else{
         print hLOG "Keep $d\n";
      }
   }
}
print hLOG "=" x 80, "\n";
print hLOG "Run END at " . `date` . "\n";
print hLOG "=" x 80, "\n";
close hLOG;

# ------------- Subroutines &#038; functions -----------------

sub get_timestamp {
   ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
   $mon+=1;

   if ($mon < 10) { $mon = "0$mon"; }
   if ($mday < 10) { $mday = "0$mday"; }
   if ($hour < 10) { $hour = "0$hour"; }
   if ($min < 10) { $min = "0$min"; }
   if ($sec < 10) { $sec = "0$sec"; }
   $year=$year+1900;

   return $year . '_' . $mon . '_' . $mday . '__' . $hour . '_' . $min . '_' . $sec;
}

sub dirIsOlder{
   ($fyear, $fmonth, $fday) = @_;
   #print "Check $year - $month - $day \n";

   $now  = mktime(localtime());
   $then =  mktime (0, 0 , 0, $fday-1, $fmonth-1, $fyear-1900 , 0, 0);

   $diff_sec = $now - $then;
   $days_since = $diff_sec / 24 / 60 / 60;
   #print "n: $now. t: $then. Diff ($diff_sec) = $days_since\n";
   #print "$days_since $fyear-$fmonth-$fday\n";

   ## Subtract days from today

   if ($days_since > $KEEP_DAYS){
      return 1;
   }else{
      return 0;
   }
}
</pre>
<h2  class="related_post_title">Related Posts</h2><ul class="related_post"><li><a href="http://www.eddieoneverything.com/linux/using-rsync-ssh-to-backup-files-between-linux-machines-perl.php" title="Using rsync &#038; ssh to Backup Files Between Linux Machines (Perl)">Using rsync &#038; ssh to Backup Files Between Linux Machines (Perl)</a></li><li><a href="http://www.eddieoneverything.com/linux/argument-list-too-long-from-rm-command-a-perl-script-to-get-around-the-problem.php" title="&#8220;Argument list too long&#8221; from rm command.   A Perl Script to Get Around the Problem">&#8220;Argument list too long&#8221; from rm command.   A Perl Script to Get Around the Problem</a></li><li><a href="http://www.eddieoneverything.com/articles/managing-server-load-when-unzipping-many-large-files.php" title="Managing Server Load when unzipping many large files">Managing Server Load when unzipping many large files</a></li><li><a href="http://www.eddieoneverything.com/articles/how-to-scan-a-computer-for-open-ports.php" title="How to Scan a Computer for Open Ports">How to Scan a Computer for Open Ports</a></li><li><a href="http://www.eddieoneverything.com/articles/the-zen-of-taco-bell-programming-using-unix-tools-to-prevent-reinventing-the-wheel.php" title="The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel">The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel</a></li><li><a href="http://www.eddieoneverything.com/articles/linux-forward-file-not-working-make-sure-you-have-the-permissions-set-correctly.php" title="Linux .forward file not working?  Make sure you have the permissions set correctly">Linux .forward file not working?  Make sure you have the permissions set correctly</a></li><li><a href="http://www.eddieoneverything.com/linux/caution-filename-not-matched-error-when-unzipping-multiple-files-workaround.php" title="Caution: filename not matched error when unzipping multiple files workaround">Caution: filename not matched error when unzipping multiple files workaround</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.eddieoneverything.com/linux/rotating-backup-directories-using-cp-al-hardlinks-to-save-disk-space.php/feed</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Using rsync &amp; ssh to Backup Files Between Linux Machines (Perl)</title>
		<link>http://www.eddieoneverything.com/linux/using-rsync-ssh-to-backup-files-between-linux-machines-perl.php</link>
		<comments>http://www.eddieoneverything.com/linux/using-rsync-ssh-to-backup-files-between-linux-machines-perl.php#comments</comments>
		<pubDate>Sun, 15 Nov 2009 19:27:32 +0000</pubDate>
		<dc:creator>eddie</dc:creator>
				<category><![CDATA[linux]]></category>
		<category><![CDATA[perl]]></category>
		<category><![CDATA[programming]]></category>
		<category><![CDATA[backup]]></category>
		<category><![CDATA[code]]></category>
		<category><![CDATA[rsync]]></category>
		<category><![CDATA[script]]></category>
		<category><![CDATA[scripts]]></category>
		<category><![CDATA[system administration]]></category>
		<category><![CDATA[ubuntu]]></category>

		<guid isPermaLink="false">http://www.eddieoneverything.com/?p=342</guid>
		<description><![CDATA[This script performs a network backup using unix&#8217;s rsync command over ssh.  It performs an incremental backup, which means that it only updates new files and files that have changed since the last backup, thus minimizing network traffic.  
This script runs on the source machine (SOURCE) and connects to the backup machine (BACKUP). [...]]]></description>
			<content:encoded><![CDATA[<p>This script performs a network backup using unix&#8217;s rsync command over ssh.  It performs an incremental backup, which means that it only updates new files and files that have changed since the last backup, thus minimizing network traffic.  </p>
<p>This script runs on the source machine (SOURCE) and connects to the backup machine (BACKUP).  (SOURCE->BACKUP). It assumes a user named backup-SOURCE on the backup machine and connects as that user. </p>
<p>One advantage of using SOURCE->BACKUP setup rather than a BACKUP->SOURCE method is that if your BACKUP server is compromised, the attacker will not be able to gain access to all of your production machines.</p>
<p>There are admittedly disadvantages to running a SOURCE->BACKUP setup, rather than a BACKUP->SOURCE setup.  One, you&#8217;ll need a copy of the script on each of the machines that you want to backup, making maintenance a bit more involved.  Second, if your SOURCE machine becomes compromised, the attacker will be able to gain access to your backup server. This second issue can be mitigated by setting the permissions properly on the BACKUP machine.  (This is why I use a separate user account for each machine that I backup.)</p>
<p>How to use this script:</p>
<ol>
<li>Edit the configuration options in the script.</li>
<li>Add a user called &#8220;backup-{$SOURCE} to the BACKUP machine.</li>
<li>Set up the appropriate SSH keys for the user to bypass password prompts.</li>
<li>Set up a cron job to run the script nightly.</li>
<li>If you want Apple-ish &#8220;time machine style&#8221; backups, set up a script to rotate the backups on the BACKUP machine.  I use this <a href="http://www.eddieoneverything.com/linux/rotating-backup-directories-using-cp-al-hardlinks-to-save-disk-space.php">rotate_backups.pl</a> script to make backups using cp -al (hardlinks).</li>
</ol>
<pre name="code" class="pl">

#!/usr/bin/perl
# Filename: rsync_backup.pl
# Will back things up from one computer to another over SSH using rsync.
# Use in conjunction with a rotate script on the destination machine for
# time-machine-like incremental backups.
# Note: SSH SSL auth keys must be installed on each end to avoid passwd prompt
# Assumes a user named "backup-$THIS_COMPUTER" on remote machine.
# All output will go to Logfile
#
# (c) 2009 eddie@eddieoneverything.com 

use Data::Dumper;

# This computer's name.  Used to name the backup files.
$THIS_COMPUTER = "yoakam";

# The remote machine's URL or IP address.
$REMOTE_MACHINE = "127.0.0.1";

# The backup directory on the remote machine
$REMOTE_BACKUP_LOCATION="/mnt/backup";

# Log things
$LOGFILE = "/var/log/rsync_backup";

# Directories to backup.
@BACKUP_DIRS=(
   '/home',
   '/root',
   '/etc',
   '/var/log',
   '/var/lib/mysql',
   '/var/spool/cron'
);

# Directories to ignore.
@IGNORE_DIRS=(
   '/home/lost+found',
   '/home/tmp',
   '/home/virtfs'
);

#####################################################
########### END Configuration options ###############
#####################################################

open hLOG, "&gt;&gt;$LOGFILE";
$now = `date`;
print hLOG "-" x 80, "\n";
print hLOG "START BACKUP $THIS_COMPUTER at $now\n" ;
print hLOG "-" x 80, "\n";
# Get a list of dirs to back up
foreach (@BACKUP_DIRS){
   $dir = $_;
   opendir(DIR, $dir) || alert ("cant opendir $dir: $!");
   while ($file = readdir(DIR)){
      if ($file ne '.' &amp;&amp; $file ne '..'){
         $my_file = $dir . "/" .  $file;
         if ( -d $my_file){
            if (grep {/$my_file/} @IGNORE_DIRS){
               print hLOG "Ignore Dir: $my_file\n";
            }else{
               print hLOG "Dir: $my_file\n";
               push @BACKUP_LIST, {'dir' =&gt; $dir, 'name'=&gt;$file};
            }
         }else{
            print hLOG "file is $my_file -- \n";
            @arr_temp =  ($dir, $my_file);
            push @BACKUP_LIST, {'dir' =&gt; $dir, 'name'=&gt;$file};
         }
      }
   }
   closedir DIR;
}

#print Dumper(@BACKUP_LIST);

#Back up each dir in the backup list
foreach $my_entry (@BACKUP_LIST){
   $dir = $my_entry-&gt;{'dir'};
   $b = $my_entry-&gt;{'name'};

   $cmd = "rsync -avz --delete-after -e ssh \"$dir/$b\" backup-$THIS_COMPUTER\@$REMOTE_MACHINE:\"/$REMOTE_BACKUP_LOCATION/$THIS_COMPUTER$dir\"";
   $now=`date`;
   print hLOG "\n$now\t$cmd ";
   #print "$cmd\n";
   `$cmd &gt;&gt; $LOGFILE 2&gt;&amp;1`;
}

$now = `date`;
print hLOG "-" x 80, "\n";
print hLOG "END BACKUP of $THIS_COMPUTER at $now\n" ;
print hLOG "-" x 80, "\n";
close hLOG;

###################### Subs #########################
sub alert(){
   $a=shift(@_);
   print hLOG "#############################\n";
   print hLOG "#############################\n";
   print hLOG "#############################\n";
   print hLOG "Cannot backup $a $!\n";
   print hLOG "#############################\n";
   print hLOG "#############################\n";
   print hLOG "#############################\n";
}</pre>
<h2  class="related_post_title">Related Posts</h2><ul class="related_post"><li><a href="http://www.eddieoneverything.com/linux/rotating-backup-directories-using-cp-al-hardlinks-to-save-disk-space.php" title="Rotating Backup Directories using cp -al (hardlinks) to Save Disk Space">Rotating Backup Directories using cp -al (hardlinks) to Save Disk Space</a></li><li><a href="http://www.eddieoneverything.com/articles/managing-server-load-when-unzipping-many-large-files.php" title="Managing Server Load when unzipping many large files">Managing Server Load when unzipping many large files</a></li><li><a href="http://www.eddieoneverything.com/linux/argument-list-too-long-from-rm-command-a-perl-script-to-get-around-the-problem.php" title="&#8220;Argument list too long&#8221; from rm command.   A Perl Script to Get Around the Problem">&#8220;Argument list too long&#8221; from rm command.   A Perl Script to Get Around the Problem</a></li><li><a href="http://www.eddieoneverything.com/articles/how-to-scan-a-computer-for-open-ports.php" title="How to Scan a Computer for Open Ports">How to Scan a Computer for Open Ports</a></li><li><a href="http://www.eddieoneverything.com/articles/the-zen-of-taco-bell-programming-using-unix-tools-to-prevent-reinventing-the-wheel.php" title="The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel">The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel</a></li><li><a href="http://www.eddieoneverything.com/programming/how-to-easily-install-a-perl-module-in-ubuntu-linux.php" title="How to Easily Install a Perl Module In Ubuntu Linux">How to Easily Install a Perl Module In Ubuntu Linux</a></li><li><a href="http://www.eddieoneverything.com/articles/linux-forward-file-not-working-make-sure-you-have-the-permissions-set-correctly.php" title="Linux .forward file not working?  Make sure you have the permissions set correctly">Linux .forward file not working?  Make sure you have the permissions set correctly</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.eddieoneverything.com/linux/using-rsync-ssh-to-backup-files-between-linux-machines-perl.php/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Calculate the Days Since a Given Date (Javascript)</title>
		<link>http://www.eddieoneverything.com/programming/calculate-the-days-since-a-given-date-javascript.php</link>
		<comments>http://www.eddieoneverything.com/programming/calculate-the-days-since-a-given-date-javascript.php#comments</comments>
		<pubDate>Sat, 14 Nov 2009 23:12:28 +0000</pubDate>
		<dc:creator>eddie</dc:creator>
				<category><![CDATA[Calculators]]></category>
		<category><![CDATA[javascript]]></category>
		<category><![CDATA[programming]]></category>
		<category><![CDATA[code]]></category>

		<guid isPermaLink="false">http://www.eddieoneverything.com/?p=258</guid>
		<description><![CDATA[I recently needed to calculate the number of days that had elapsed since a given date.  Not happy with any of the code snippets I found on the internet, I wrote my own.  A demo is below, along with the the javascript code.  Note that the code uses a bit of dynamic [...]]]></description>
			<content:encoded><![CDATA[<p>I recently needed to calculate the number of days that had elapsed since a given date.  Not happy with any of the code snippets I found on the internet, I wrote my own.  A demo is below, along with the the javascript code.  Note that the code uses a bit of dynamic HTML so no page refresh is necessary.  The answer will appear beneath the form.</p>
<p>Feel free to use these in your projects.</p>
<div class="q">
<h3>Demo &#8211; Calculate the number of days since a given date</h3>
<p><script type="text/javascript" src="/js/daysSince.js"></script></p>
<form id="theForm" name="theForm" method=POST>
<br />Day<br />
<input type=text name=day id="day">
<br />Month<br />
<input type=text name=month id="month">
<br />Year<br />
<input type=text name=year id="year">
<input type=button onclick='daysSince(document.getElementById("month").value,document.getElementById("day").value,document.getElementById("year").value)' value ="Calculate days since">
<p>
<div id="result"></div>
</form>
</div>
<p>To get this to work on your website, you&#8217;ll need the following javascript function, along with the HTML file below.</p>
<p>The JavaScript:</p>
<pre name="code" class="javascript">
function daysSince(m,d,y){
   if (d == '' || m=='' || y==''){
      alert ("All fields must be entered");
      return;
   }

   if (isNaN(m) || isNaN(y) || isNaN(d)){
      alert("Only numbers .");
      return;
   }

   var myDate=new Date();
   var yourDate=new Date(y,m-1,d);

   var secondsInADay = 1000*60*60*24;
   var diff = Math.floor( (myDate.getTime() - yourDate.getTime()) / secondsInADay );
   document.getElementById("result").innerHTML= "It has been " +
      diff + " days since " + m + "/" + d + "/" + y +".";
}
</pre>
<p>The HTML:</p>
<pre name="code" class="HTML">
<form action ="" id="theForm" name="theForm">
Day
<input type=text name=day id="day">
Month
<input type=text name=month id="month">
Year
<input type=text name=year id="year">
<input type=button onclick='daysSince(document.getElementById("month").value,document.getElementById("day").value,document.getElementById("year").value)' value ="Calculate days since">
<div id="result"></div>
</form>
</pre>
<p>If you make any useful alterations, feel free to share them in the comments.</p>
<h2  class="related_post_title">Related Posts</h2><ul class="related_post"><li><a href="http://www.eddieoneverything.com/articles/how-to-scan-a-computer-for-open-ports.php" title="How to Scan a Computer for Open Ports">How to Scan a Computer for Open Ports</a></li><li><a href="http://www.eddieoneverything.com/articles/the-zen-of-taco-bell-programming-using-unix-tools-to-prevent-reinventing-the-wheel.php" title="The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel">The Zen of Taco Bell Programming &#8211; Using Unix Tools to Prevent Reinventing the Wheel</a></li><li><a href="http://www.eddieoneverything.com/programming/how-to-use-global-variables-from-another-file-in-a-perl-module-or-package.php" title="How To Use Global Variables From Another File In A Perl Module or Package">How To Use Global Variables From Another File In A Perl Module or Package</a></li><li><a href="http://www.eddieoneverything.com/programming/getting-phps-print_r-function-to-return-a-string.php" title="Getting PHP&#8217;s print_r Function To Return A String">Getting PHP&#8217;s print_r Function To Return A String</a></li><li><a href="http://www.eddieoneverything.com/programming/how-to-set-a-class-name-using-javascript-in-ie-and-other-browsers.php" title="How To Set a Class Name using Javascript in IE and Other Browsers">How To Set a Class Name using Javascript in IE and Other Browsers</a></li><li><a href="http://www.eddieoneverything.com/programming/calibre-recipe-for-minneapolis-star-tribune-startribune-com.php" title="Calibre Recipe for Minneapolis Star Tribune (StarTribune.com)">Calibre Recipe for Minneapolis Star Tribune (StarTribune.com)</a></li><li><a href="http://www.eddieoneverything.com/linux/argument-list-too-long-from-rm-command-a-perl-script-to-get-around-the-problem.php" title="&#8220;Argument list too long&#8221; from rm command.   A Perl Script to Get Around the Problem">&#8220;Argument list too long&#8221; from rm command.   A Perl Script to Get Around the Problem</a></li></ul>]]></content:encoded>
			<wfw:commentRss>http://www.eddieoneverything.com/programming/calculate-the-days-since-a-given-date-javascript.php/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

