What I learned from GSoC?

Google Summer of Code was a wonderful experience for me, I’ll never forget it. I finished my project and that makes me feel great. Getting opinions and feedback from the Open Source community was really special. In general, this is what I learned:


Improved my communication skills: Doing a project that involves other people means that you have to be clear in your words, be brief but efficient, and most important, make people understand what you are trying to say.


I wrote a lot in English. (My project, IRC meetings, asking in the list, answering questions, etc). Although, English is not my first language I enjoyed having to be clear with my mentor, the OSVDB (Open Sourced Vulnerability Database) developers and the Google group of summer of coders.


Ruby and Ruby on Rails: This Ruby user’s guide helped me a lot. I haven’t finished it, but I learned a lot about this great language (regular expressions, strings, arrays, iterators, control structures, OOP, classes, methods). And I also do a lot of programing with the ruby on rails framework. I learned how to manage and modified some plugins.

Actually, working with views is not my favorite part of a project. However, CSS work at the end of the project was really fun. What CSS makes is impressive.

Solr: I never ever have worked with a search server. Solr is just amazing – Fulltext search capabilities. I learned how to integrate Solr and the act_as_solr plugin to my Rails application.

Subversion: I learned how to work with this collaboration tool which I consider makes you more productive. However, I had to deal with so many error messages. I need to learn more commands apart from the common ones.

Vulnerability and Patch concepts.

I believe that being an expert in these fields takes his time. I never did a security system and didn’t work with a security team before. But I learned key concepts related to patches and vulnerabilities that helped me a lot to write the code. For example:

  • Vulns classification: Location, Attack Type , Impact, Solution, Exploits.
  • Vulns technical description and how to test a vulnerability.
  • Patch severity: Critical, Severe,important, Minor, Pointless.
  • Security Products: Nikto, Snort and Nessus.
  • What CVE means.
  • How to associate a vuln-patch with a Vendor/Product/Version.

Many lessons learned in these months let me think what I did wrong and what I did well. I feel that  I’m not always as productive as I might like to, my effort changes with the tasks I’m doing.

Once I read some Linus interview in which he said that if you are completely present in a situation and totally focused on something then that something *becomes* interesting, whatever it may be.

So, I think that making your job interesting and fun and get really focused on the problem are the keys to your project success.

Thanks for your visit to the blog. You can follow me on Twitter:

¡Happy Hacking!

Rails 2.2 will be thread safe

Reading the rubyonrails blog I read that Rails 2.2 will be thread safe and that this hard work comes from a Google Summer of Coder. Josh Peek who will join the rails core.

This is a great advance for the framework, because many people were looking for merb for this rails weak, but this improvement will attract new developers to join the rails community.

Congratulations Josh.
Thanks for your visit to the blog. You can follow me on Twitter:


Rdoc – Generating documentation for Ruby and Rails

Rdoc is a program for creating documentation for ruby source code. Rdoc generates HTML documentation, using syntactic information from the source and text in comment blocks.

You can check documentation about ruby libraries if you have ruby property installed by running the ri command. For example, if we want to know what the capitalize method does, just type in your terminal:

ri String.capitalize

str.capitalize    => new_str
Returns a copy of str with the first character converted to
uppercase and the remainder to lowercase
"hello".capitalize  #=> "Hello"
"HELLO".capitalize  #=> "Hello"
"123ABC".capitalize  #=> "123abc"

Rdoc is very useful to look for information about programs, methods and examples.

If you want to see the rails api documentation type in your terminal: gem server and then go to your browser at the url http://localhost:8808

You will get an HTML page “RubyGems Documentation Index” with a summary of all ruby gems installed on your system.

Click on the gem you want to see the documentation and you will see an HTML page with 3 columns: (Files, Classes, Methods) and everything documented in detail.

If you installed rails with rubygems you can access the rails API. Also the rake app:doc command creates the HTML documentation for your Rails project and stores this documentation in the doc/app directory.

Run the file  doc/app/index.html and you will see of the documentation of your rails project.
Thanks for your visit to the blog. You can follow me on Twitter:

Query analyzer for Rails

I found an interesting and very useful plugin for rails called Query analyzer that allows us check for tables not optimized in our database. It really helps a lot to optimize tables and queries and put indexes on our conditions columns and primary/foreign keys. Sometimes, we forget to do things so simple like this.  Here is a great explanation of what the plugin does.

How to use it? Just install the plugin with:

script/plugin install http://svn.nfectio.us/plugins/query_analyzer

Then check your logs. You can see your console as well. Once you have the plugin installed you will see all queries your application does and how they are currently managed.

For example:

I have a table called form_help_divs. It is a table to show some messages in forms around my application. I usually make a call to the database to get a form_help_div record. The query generated is like this:

SELECT * FROM `form_help_divs` WHERE (`form_help_divs`.`name` = 'patch_vuln') LIMIT 1

So in theory. The query should check for only one row, but that is not true because the name column doesn’t have an index. And how I realized that?. I just checked my log and saw this:

Analyzing FormHelpDiv Load

select_type | key_len | type | Extra       | id | possible_keys | rows | table          | ref | key
SIMPLE      |         | ALL  | Using where | 1  |               | 33   | form_help_divs |     |

As you see the query is looking for all records (in this case 33) in the database searching for the name ‘patch_vuln’. That is not good guys. Let us optimize our queries by adding and index to that table. We can add it by hand in our mysql interface, but is better to create migrations.

class AddFormHelpDivIndexes < ActiveRecord::Migration
    def self.up
        add_index :form_help_divs, :name

    def self.down
        remove_index :form_help_divs, :name

Type rake db:migrate in your console and your migration will be executed.Now run your application again and you will see the difference in your logs and console. This is the result:

FormHelpDiv Load (0.001585)   SELECT * FROM `form_help_divs` WHERE (`form_help_divs`.`name` = 'patch_vuln') LIMIT 1

Analyzing FormHelpDiv Load

select_type | key_len | type | Extra     | id | possible_keys | rows |table  | ref   | key
SIMPLE   |768|ref|Using where|1|name,index_form_help_divs_on_name|1|form_help_divs | const | name

¡What a big difference!. Now the query is looking just for one row in the database and that will increase the query speed significantly.

There are other ways to make our applications running faster with rails. Adding indexes is one of those ways and let us gain a lot of optimization specially if we are working with big applications and tables with so many records.
Thanks for your visit to the blog. You can follow me on Twitter:


Rake is a Ruby program that builds other Ruby programs.  Each time you execute rake, it knows how to build those programs by reading a file called Rakefile which has a set of tasks.  Those tasks allows us to do some project needs in a very easy and efficient way.

When you generate a rails project you automatically get a Rakefile and it is located in the root of your project.

You can see all rake tasks and their descriptions by running a simple command in your main directory:

rake –tasks

And this should be shown:

 rake backgroundrb:remove # Remove backgroundrb from your rails ...
 rake backgroundrb:restart # Restart backgroundrb server (default...
 rake backgroundrb:setup # Setup backgroundrb in your rails app...
 rake backgroundrb:start # Start backgroundrb server (default v...
 rake backgroundrb:stop # Stop backgroundrb server (default va...
 rake db:abort_if_pending_migrations # Raises an error if there are pending...
 rake db:charset # Retrieves the charset for the curren...
 rake db:collation # Retrieves the collation for the curr...
 rake db:create # Create the database defined in confi...
 rake db:create:all # Create all the local databases defin...
 rake db:drop # Drops the database for the current R...
 rake db:drop:all # Drops all the local databases define...
 rake db:fixtures:identify # Search for a fixture given a LABEL o...
 rake db:fixtures:load # Load fixtures into the current envir...
 rake db:migrate # Migrate the database through scripts...
 rake db:migrate:redo # Rollbacks the database one migration...
 rake db:migrate:reset # Resets your database using your migr...
 rake db:reset # Drops and recreates the database fro...
 rake db:rollback # Rolls the schema back to the previou...
 rake db:schema:dump # Create a db/schema.rb file that can ...
 rake db:schema:load # Load a schema.rb file into the database
 rake db:sessions:clear # Clear the sessions table
 rake db:sessions:create # Creates a sessions migration for use...
 rake db:structure:dump # Dump the database structure to a SQL...
 rake db:test:clone # Recreate the test database from the ...
 rake db:test:clone_structure # Recreate the test databases from the...
 rake db:test:prepare # Prepare the test database and load t...
 rake db:test:purge # Empty the test database
 rake db:version # Retrieves the current schema version...
 rake doc:app # Build the app HTML Files
 rake doc:clobber_app # Remove rdoc products
 rake doc:clobber_plugins # Remove plugin documentation
 rake doc:clobber_rails # Remove rdoc products
 rake doc:plugins # Generate documentation for all insta...
 rake doc:rails # Build the rails HTML Files
 rake doc:reapp # Force a rebuild of the RDOC files
 rake doc:rerails # Force a rebuild of the RDOC files
 rake log:clear # Truncates all *.log files in log/ to...
 rake notes # Enumerate all annotations
 rake notes:fixme # Enumerate all FIXME annotations
 rake notes:optimize # Enumerate all OPTIMIZE annotations
 rake notes:todo # Enumerate all TODO annotations
 rake rails:freeze:edge # Lock to latest Edge Rails or a speci...
 rake rails:freeze:gems # Lock this application to the current...
 rake rails:unfreeze # Unlock this application from freeze ...
 rake rails:update # Update both configs, scripts and pub...
 rake rails:update:configs # Update config/boot.rb from your curr...
 rake rails:update:javascripts # Update your javascripts from your cu...
 rake rails:update:scripts # Add new scripts to the application s...
 rake remove_simple_captcha_files # Remove unuseful captcha images and s...
 rake routes # Print out all defined routes in matc...
 rake secret # Generate a crytographically secure s...
 rake solr:destroy_index # Remove Solr index
 rake solr:start # Starts Solr.
 rake solr:stop # Stops Solr.
 rake stats # Report code statistics (KLOCs, etc) ...
 rake test # Test all units and functionals
 rake test:functionals # Run tests for functionalsdb:test:pre...
 rake test:integration # Run tests for integrationdb:test:pre...
 rake test:plugins # Run tests for pluginsenvironment / R...
 rake test:recent # Run tests for recentdb:test:prepare ...
 rake test:uncommitted # Run tests for uncommitteddb:test:pre...
 rake test:units # Run tests for unitsdb:test:prepare /...
 rake tmp:cache:clear # Clears all files and directories in ...
 rake tmp:clear # Clear session, cache, and socket fil...
 rake tmp:create # Creates tmp directories for sessions...
 rake tmp:pids:clear # Clears all files in tmp/pids
 rake tmp:sessions:clear # Clears all files in tmp/sessions
 rake tmp:sockets:clear # Clears all files in tmp/sockets
 rake uml:schema # Generate an XMI db/schema.xml file d...

Those rake tasks will be used many times when developing your rails application.
For example, there is an interesting task rake stats which generates detailed statistics about your application code and provides a dashboard of information.  This is what I get:

| Name                 | Lines |   LOC | Classes | Methods | M/C | LOC/M |
| Controllers          |  6080 |  5241 |      54 |     369 |   6 |    12 |
| Helpers              |   552 |   501 |       0 |      25 |   0 |    18 |
| Models               |  3855 |  3274 |     157 |     326 |   2 |     8 |
| Libraries            |  3068 |  2806 |      18 |      71 |   3 |    37 |
| APIs                 |     9 |     9 |       1 |       0 |   0 |     0 |
| Components           |     0 |     0 |       0 |       0 |   0 |     0 |
| Integration tests    |     0 |     0 |       0 |       0 |   0 |     0 |
| Functional tests     |  1261 |   942 |      97 |     200 |   2 |     2 |
| Unit tests           |  1246 |   884 |     117 |     129 |   1 |     4 |
| Total                | 16071 | 13657 |     444 |    1120 |   2 |    10 |
Code LOC: 11831     Test LOC: 1826     Code to Test Ratio: 1:0.2

There are many other rake commands very useful for rails development.

Thanks for your visit to the blog. You can follow me on Twitter:

Hpricot installation problem

Hpricot is a fast,  flexible HTML parser written in C. Hpricot can be handy for reading broken XML files,  since many of the same techniques can be used. If a quote is missing Hpricot will try to figure it out.

I was really having problems to install hpricot,  the newest version is 0.6.  Whenever I executed:

$ sudo gem install hpricot
Select which gem to install for your platform (arm-linux)
1. hpricot 0.6 (mswin32)
2. hpricot 0.6 (jruby)
3. hpricot 0.6 (ruby)
4. hpricot 0.5 (ruby)
5. hpricot 0.5 (mswin32)
6. Skip this gem
7. Cancel installation
> 3

I received the following message:

Building native extensions.  This could take a while...
ERROR:  While executing gem ... (Gem::Installer::ExtensionBuildError)
ERROR: Failed to build gem native extension.

I Typed this to see what is going on:

ruby extconf.rb install hpricot

and the error continue

extconf.rb:1:in `require': no such file to load -- mkmf (LoadError)
from extconf.rb:1

Gem files will remain installed in /var/lib/gems/1.8/gems/hpricot-0.6 for inspection. Results logged to /var/lib/gems/1.8/gems/hpricot-0.6/ext/hpricot_scan/gem_make.out

Googling for a while,  I found a good solution in the hpricot project wiki.  The main problem is that I had ruby 1.8 and 1.9 installed. Ruby 1.8.5 was the version I was using, but I didn’t have development libraries installed.  So, gem couldn’t install correctly hpricot.

You can check your version for ruby, rails, gem and everything else by typing -m after the program name.

$ ruby -v
ruby 1.8.5 (2006-08-25) [i486-linux]
$ gem -v

The solution: There is a file we need,  called mkmf.rb. It has connection with ruby libraries,  so let’s search it.

$ auto-apt search mkmf.rb
usr/lib/ruby/1.9/mkmf.rb        devel/ruby1.9-dev

As you can see, we don't have it in ruby1.8 directory, so let us install ruby1.8 development libraries.

$ sudo apt-get install ruby1.8-dev

and now, we have the mkmf.rb in ruby1.8

$ auto-apt search mkmf.rb
usr/lib/ruby/1.9/mkmf.rb        devel/ruby1.9-dev
usr/lib/ruby/1.8/mkmf.rb        devel/ruby1.8-dev

There is no documentation in the wiki,  but I assume that you have to install another gem called mechanize which is  an hpricot  dependency.

$ sudo gem install mechanize
Install required dependency hpricot? [Yn]
Select which gem to install for your platform (arm-linux)
1. hpricot 0.6 (mswin32)
2. hpricot 0.6 (jruby)
3. hpricot 0.6 (ruby)
4. hpricot 0.6 (jruby)
5. hpricot 0.6 (ruby)
6. hpricot 0.6 (mswin32)
7. Skip this gem
8. Cancel installation
> 3
Building native extensions.  This could take a while...
Successfully installed mechanize-0.6.9
Successfully installed hpricot-0.6
Installing ri documentation for mechanize-0.6.9...
Installing ri documentation for hpricot-0.6...
Installing RDoc documentation for mechanize-0.6.9...
Installing RDoc documentation for hpricot-0.6...

Finally, you will get hpricot successfully installed and now you can use this great gem.

Thanks for your visit to the blog. You can follow me on Twitter:


Rails has a great feature called migrations which allow the developer to have control over the database schema by using ruby code and avoiding use the conventional SQL language. But, more important is that the developer can apply certain changes to move a database from one state to another,  add-remove columns, indexes,  tables etc.  By using migrations you can get different versions of your database structure.

How it works? Migrations are files found in the db/migrate directory and have a sequence number in the filename.

For example, each time you want to add a migration you can generate it with:

./script/generate migration CreateCompanies

exists  db/migrate
create  db/migrate/001_create_companies.rb
Loaded suite ./script/generate
Finished in 0.002254 seconds.
0 tests, 0 assertions, 0 failures, 0 errors

If you see the file, this was generated for you:

class CreateCompanies < ActiveRecord::Migration
    def self.up


    def self.down


and now you can modify it to create a table, the syntax is the folowing:

class CreateCompanies < ActiveRecord::Migration
    def self.up
        create_table :companies do |t|
            t.string    :name
            t.text      :description
            t.datetime  :created_on

    def self.down
        drop_table :companies

I think the code is explained implicity.  It’s similar to an sql creation,  but this time you have to add the datatype first and then the column name.

Possible Column types:
:string, :text, :integer, :float, :datetime, :timestamp, :time, :date, :binary, :boolean

Column details:
:limit: Maximum long characters in the column (for types :string, :text, :binary or :integer)
:default: Specify the default value of the column.
:null: Enable or disable the value NULL in a column.

To get this table created, just you have to run the command: rake db:migrate and the companies table will be created.

Imagine you want to create another table users, generate it with:

./script/generate migration CreateUsers

exists  db/migrate
create  db/migrate/002_create_users.rb
Loaded suite ./script/generate
Finished in 0.002254 seconds.
0 tests, 0 assertions, 0 failures, 0 errors

Modify the file 002_create_users.rb

class CreateUsers < ActiveRecord::Migration
    def self.up
        create_table :users do |t|
            t.string   :login
            t.string   :email
            t.string   :password,   :limit => 20
            t.datetime :created_at
            t.datetime :updated_at
            t.integer  :status
            t.datetime :last_login

    def self.down
        drop_table :users

and if you run rake again db:migrate table users will be created in your database. You are probably wondering how rails knows next migration should be named 002 and when running rake db:migrate which migration has to be executed.

Whenever doing these operations rails checks in your database a table called schema_info, which has the latest version of your database schema.  So, that way rails knows which migration number to apply or generate.

Try to run again rake db:migrate and you will receive an error message

Mysql::Error: Table ‘users’ already exists:

telling you that table users already exists.  This is because this migration was already executed and in your schema_info table the version is 2 (the same as the number of your migration). Now you can realize what the pattern is.  Also you can execute an specific migration by running:

rake db:migrate version= version_number

version_number is the version you want to be executed.

When developing your rails application you will see that migrations are very useful because you are database-independent, can change to one version or another and undo changes easily.
Thanks for your visit to the blog. You can follow me on Twitter: