Open-sourcing Vessel: a high-level Ruby web crawling framework
The Californer/10241986

Vessel web crawling and scraping framework for Ruby. Fast as Chrome, dead simple and yet extendable.

SAN FRANCISCO - Californer -- Vessel is a fast, open-source, high-level, web crawling and scraping framework for Ruby, built on top of Ferrum — a minimal-dependency pure-Ruby driver for running headless Google Chrome instances.

Why would you need a web crawler? Perhaps you're building a search engine for an intranet or a group of public-facing websites, or just need to mirror a website with finer-grained control than tools such as wget offer.

The best way to demonstrate Vessel's capabilities is with an example. Don't worry, just because of what Vessel's capable of, it doesn't mean that it's hard to use.

You can see from the example how easy it is to scrape — extract structured data from typically-unstructured web pages — using Ferrum's DOM methods.

More on The Californer
In fact, with Vessel and Ferrum, you can crawl, parse, extract, and transform web content with so little effort, you'll wonder why you ever had to do it any other way before!

Ferrum + Crawling = Vessel

If you're considering using Ferrum to perform some web crawling, then look no further than Vessel a Ferrum-based crawling framework for Ruby.

Learn how you can use Vessel framework in your project.

Media Contact
44 Tehama St, San Francisco, CA 94105

Filed Under: Open source

Show All News | Report Violation


Latest on The Californer