The World of a difference with mysql ‘index’

I’ve created a simple mozilla extension search engine. Everytime I submitted a query, it would take forever for my suggestions to pop up. Frustrated at the latency, I looked at my log files to see this:

Processing SearchController#suggest_package (for 10.1.0.132 at 2007-06-27 18:52:32) [GET]
Session ID: 2836272b416b6f5e190f457b4c5644bb
Parameters: {“action”=>”suggest_package”, “package”=>”ray”, “controller”=>”search”}
Package Load (0.163366)   SELECT * FROM packages WHERE (p_name LIKE ‘ray%’) GROUP BY p_name ORDER BY p_name LIMIT 25
Package Columns (0.000900)   SHOW FIELDS FROM packages
Completed in 0.17745 (5 reqs/sec) | Rendering: 0.00013 (0%) | DB: 0.16427 (92%) | 200 OK [http://10.1.0.111/search/suggest_package?package=ray]
Notice anything interesting? How about the 92% DB load?

Disgusted by the results, I quickly realized that I didn’t index my queried field of 90k tuples so I created an index on that field by typing in the following into the mysql console:

create index name using btree on packages(p_name)

btw (btrees are the craziest things in the world, I got dominated in college trying to write one. I couldn’t get delete to save my life)

After creating the index, I submitted a second query and got:

Processing SearchController#suggest_package (for 10.1.0.132 at 2007-06-27 18:53:35) [GET]
Session ID: 2836272b416b6f5e190f457b4c5644bb
Parameters: {“action”=>”suggest_package”, “package”=>”ray”, “controller”=>”search”}
Package Load (0.002940)   SELECT * FROM packages WHERE (p_name LIKE ‘ray%’) GROUP BY p_name ORDER BY p_name LIMIT 25
Package Columns (0.002396)   SHOW FIELDS FROM packages
Completed in 0.01962 (50 reqs/sec) | Rendering: 0.00009 (0%) | DB: 0.00534 (27%) | 200 OK [http://10.1.0.111/search/suggest_package?package=ray]
27% ain’t that bad. I can live with 27% considering the fact that its running on a VM and im on a laptop.

Lesson to be learn. USE INDICIES!

Zlib GzipReader can be confusing

For whatever reasoning, the only way i can get the gzip reader to work is if I use this command:

package_data = “”
Zlib::GzipReader.open(“data/debian_packages_#{right_now}.txt.gz”) { |gzip|
package_data += gzip.read
}

Here is a good reference to how you can use Gzip. Unfortunately I couldn’t get any of the other examples or even my own to work. I kept on getting error: ‘Invalid char `\302′ in expression ruby’ . Whatever that means?

http://www.oreilly.com/catalog/rubyckbk/toc.html

Custom mozilla search extension

I tried to write my own plugin to do simple searches when I realized that all I needed was a search extension. Since mozilla adopted the opensearch standard, integrating your custom search hasn’t been easier. Sandro Paganotti wrote a pretty straight forward article on how to create your own custom mozilla search extension (http://www.railsonwave.com/railsonwave/2007/2/21/tutorial-opensearch-with-rails-1-2-render-json) Its actually a lot simpler than you might think.

Once implemented, don’t be alarmed if you don’t get an immediate prompt or automatic install of your newly created search extension. Those prompts are made possible through a little bit of javascript. Just look under your search bar and it should say “Add ‘your search extension name'”. Or if you want the browser to prompt you for search extension install, use the “addSearchEngine” function

addSearchEngine(engine URL, icon URL, engine name, category name)

Feeds are easy

Feeds are just glorified xml… just keep with the syntax and let the browser handle the rest. Whether its RSS2 or atom, google for the xml syntax and there you have it. Handling the different requests on the controller end is another story. But not too bad. With help of rails’s respond_to (http://rails.raaum.org/actionview.html) you can dynamically render different formats without repeating any code and adhearing to the DRY method Rails is built on.

Atom : http://snippets.dzone.com/posts/show/559

RSS2: http://snippets.dzone.com/posts/show/558

rake aborted undefined method ‘exclude’ for nil:NilClass

I’ve been developing on a remote server for the past year or so and its great when you have internet. But I’ve frequently found myself in situations where I don’t have the internet so I’ve decided to download Aptana + RDT + Subclipse (SVN for eclipse) to develop locally. So, after spending a few hours getting Aptana to work correctly, then RDT, and then subclipse I found myself unable to rebuild the openid server locally. I followed the INSTALL instructions to the dot but kept on getting “rake aborted undefined method ‘exclude’ for nil:NilClass” as an error when I tried to do a migration. As it turns out, there is something funky with rake version 0.7.3 as pointed out by Brian in  http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/248953 . I followed his advise in modifying the rake clean code and BAM, the migration worked. Little things like this is driving me crazy!

ah! Ubuntu Java updates

Installing java is pretty simple, just update your source list in /etc/apt/sources.list and then launch Synaptic package manager and pick the new version of java… BUT, How you change the system’s path to launch your new version of java?
sudo update-alternatives –config java

Ah!!! I mean, seriously, it should have updated itself somehow knowing that I just installed a new version of java.