January 26, 2008

Test post

Testing spam filter

Posted by pete at 4:50 PM | Comments (0) | TrackBack (0)

February 9, 2007

Finding array intersections faster in Perl (improving on Perl Cookbook)

Before I start, some background is in order since the term intersection isn't necessarily a common word.

The intersection between two sets (lists, arrays, etc), say A = {1,2,3} and B = {2,3,4} (written A ∩ B) is the items that are common between the two. In this case, A ∩ B = {2,3}.

Some code I'm modifying uses an intersection to find the common items after a sequence of filters is applied (i.e. each filter removes non-matching items from the original list, until we're left with the elements that matched every filter). The implementation used is one recommended in the Perl Cookbook, recipe 4.8:

@a = (1,2,3);
@b = (2,3,4);
%isect = %union = ();
foreach $e (@a, @b) { $union{$e}++ && $isect{$e}++ }
@isect_list = keys %isect

This works quite well, but I didn't need to find the union, and it requires no duplicate items within a list (there is another version of the recipe that would work with dups, but still requires tracking the union to find the intersection).

I came up with a different recipe. This is specific for intersection and doesn't assume uniqueness in the lists:

@a = (1,2,3);
@b = (2,3,4);
%isect = ();
map { $isect{$_} = 1 } @a
@isect_list = grep { $isect{$_} } @b;

As a bonus, benchmarking this recipe vs the one above shows a 3x performance increase.

If you needed to find the union (all items present in either list), this is what it would look like:

@a = (1,2,3);
@b = (2,3,4);
%union = ();
map { $union{$_} = 1 } @a, @b;
@union_list = keys %union;

This also has a 3x performance increase over the recipe.

Posted by pete at 4:20 PM | Comments (1)

February 5, 2007

Several ways to find "end of yesterday" in MySQL

Today I needed to find the time for "the end of yesterday" (i.e. 11:59:59 pm yesterday). MySQL has a handy month-end function LAST_DAY() which "takes a date or datetime value and returns the corresponding value for the last day of the month." Doesn't look like there's an equivalent built-in function for end-of-day, so I came up with a few different ways:

  • (now() - interval time_to_sec(time(now()))+1 second)
  • concat(date(now() - interval 1 day),' 23:59:59')
  • subtime(now(),time(now())) - interval 1 second

I went with the last one, it seemed the most straight-forward. If there's a better way, I'd love to hear about it.

Posted by pete at 2:07 PM

January 24, 2007

On learning to program (in 21 days or 10 years)

A few years ago Peter Norvig wrote a great article about how absurd it is —contrary to many popular titles—to learn to program well (or learn most other worthwhile skills) in less than several (ten) years.

I first came across this article when Phil Windley mentioned it a few weeks ago (I wish he'd written more about his perspective as a CS teacher). It got me thinking about my programming roots, and the several times I've taught programming to others. I realized I really haven't thought a lot about my programming background.

I started programming as a kid 25 years ago, on the Apple ][+ at the local public library. I spent a lot (probably too much) of my free time programming (AppleSoft BASIC, TRS-80 BASIC, 6502 assembler, Pascal, C) over 2-3 years, and off-and-on through my teens until I got my first full-time job as a programmer. Even though I haven't been a full-time programmer for most of those 25 years, I've easily programmed for a cumulative 10 years. Wow.

The thought process this article started helped me realize a few things (that may apply to other programmers):

  • I think about programming more than the average person (maybe more than the average programmer). Even when I've worked in other jobs, I tend to see many problems as programming opportunities (and can't understand how anyone could live with tedious manual processes). I've always been a "programmer-mailroom-sorter" or "programmer-network-engineer" or "programmer-NOC-manager" or "programmer-IT-manager".
  • I forgot a long time ago how I learned to program. My brain has been wired for "programmer think" for a long time, so I understand why teaching someone else to program and think like a programmer is a different effort than I'd expected.
  • Programming is something like riding a bike: the basic skills and thought processes that are tough to learn seem to stick with you, though they may need to be fine-tuned to the latest technologies.

    Probably a good thing I've gone back to programming full-time.

    Posted by pete at 12:54 PM

    January 20, 2007

    Updates on recent posts

    A few updates on recent posts:

  • Joe Hewitt, the creator of FireBug, wrote an excellent article in DDJ on using FireBug. His focus is on debugging AJAX, but there's useful information for any Web developer. Finding FireBug continues to be an "a ha!" moment for Web developers. If you're not using it, you're spending way more time than you should debugging and fiddling around with CSS, JavaScript and HTML.

  • After writing about algorithms and Perl, I stumbled across Mastering Algorithms with Perl (author interview). I looked through it at a local university library and it's on my to-buy list now. Not only does it have an in-depth look at sorting in Perl, there's in-depth coverage of all the basic data structures (lists, trees, heaps), searching, sets, matrices, graphs, strings, etc, etc, etc. In the 30 minutes I looked at it, I found several great tips I could put to use. Looks like a worthy addition to the reference library.

    Posted by pete at 8:08 AM

    •  
    Golf Tips