To make it easier to use the Helioid API we have released open source Ruby and Clojure client libraries. Install the Ruby library with:

gem install heliapi

then load and fetch categories using:

require 'heliapi'

results = Heliapi.new.web('ruby apis')

results['categories'].keys

which returns:

=> ['Developer',
    'Access',
    'Provides',
    'Rails',
    'Building',
    'Install',
    'Google Api Ruby',
    'Ruby Client'
]

To install the Clojure library add heliapi to your Leiningen project.clj file:

[heliapi "0.0.1"]

then load and fetch categories with:

(:require [heliapi.core :as helioid])

(map #(:name %)
     (:categories (helioid/web "helioid")))

which returns the results as:

=> ("search refinement"
    "search engine"
    "results"
    "helioid choroiditis"
    "intranuclear helioid inclusions"
    "intranuclear helioid"
    "new"
    "helioid search")

We will add features to the API and client libraries as requested. We will also make libraries for other languages as requested.

We have put online an alpha version of a JSON API for Helioid’s search results. Test it out, see what works and what could be better, then let us know. We are going to iterate on it to find the best structure.

For example, the search results for Data Analytics can be returned as JSON using this query: http://www.helioid.com/searches/q/Data+Analytics?format=json.

The returned results are stuctured as follows:

{
  "query": "Data Analytics",
  "date": "2013-10-27T03:15:53Z",
  "categories": [{
    "name": "data mining",
    "items": [{
      "rank": 1,
      "summary": "Gain fresh new insights into emerging trends and
                  behaviors that can increase your revenue and reduce
                  cost with Teradata <b>data</b> mining technology.",
      "title": "Teradata - Data Mining and Analytics Solutions",
      "url": "http://www.teradata.com/business-needs/..."
    },
    ...
    ]
  ...
  }]
}

The results first include details about the query and then a map from each category name to the set of search results (“items”) within that category.

It’s been a while since our last update, indicating a transition for the Helioid team, and we thought it was time for an update on what we’ve been up to. As we’ve noted in many past posts, our dedication to Helioid came from our own frustration in exploring new topics of interest and a desire to provide a tool that helped filter out the proverbial needles from the proverbial haystacks by categorizing search results. We remain committed to helping that long tail of exploratory searchers and to working with and supporting the community of innovators in the search space.

So, we’re pleased to announce the release of a categorization library, installable as a Ruby gem, which developers can use to categorize search results, or any other document collections they’d like to add a layer of structure to, in order to support more intuitive navigation. For example, the cluster model categorizes the first five summaries for a “helioid” search as:

“search refinement”

“floor silicon”

“word difficult”

These could use some work.

The gem includes usage examples and a walkthrough of how to customize the gem to your needs. Developers can apply categorize using alternative clustering models, and the gem includes a simple bag-of-words clustering algorithm, as well as both hierarchical and non-hierarchical Ward clustering based algorithms. Developers can customize the basic parameters of whichever categorization model they use, such as the maximum number of clusters, and the minimum support threshold for clustering. We’re excited to see the range of sites and alternative search tools the developer community applies Helioid’s categorization to using this library.

A few nights ago, after finally getting around to watching as much of Mitt Romney’s RNC speech as I could bear, I had a curious dream. In this dream, I came upon a lake covered in a haze, in the dim light of the wee hours. And beside this lake, I found a picnic table. And at that picnic table, sat Mitt Romney. I sat down, we began to chat, and I was pleased to find he was much more personable than I would have guessed. After a while, he opened up and confided that he was coming to terms with the fact that he probably wasn’t going to be President. I agreed that the odds were against him, and admitted that I myself was coming to terms with an impending transition.

We’ve been working for some time to establish Helioid as a self sufficient search company, aiming to succeed where many startups in this space have failed, by serving the long tail of exploratory searchers and offering advanced research tools to heavy users. We’ve had considerable success: we’ve provided a novel tool serving thousands of unique users a day, and from conversations we’ve had with the people who’ve tried us out, it’s clear we’ve hit upon a pain point that has gone under-addressed in search.

But for a variety of reasons, the Helioid team has decided to refocus our efforts toward other interests of ours outside of search, and hold off on further major development of Helioid Search. The good news, however, is that we will maintain our present search services, and of course we will keep you all up to date on our ongoing projects. Thank you to all of our dedicated users, and we hope you keep using Helioid to suss out the needles from those haystacks. And thanks to Dream Mitt Romney, for listening.

Helioid is committed to testing and proving our approaches to search against information retrieval problems affecting a wide range of research-oriented fields. As such, Helioid was a participant in the TREC 2011 Legal Track challenge, all submissions and results of which are now public.

The TREC 2011 Legal Track challenged participants to rank and score a large corpus of emails seized by the Federal Regulatory Energy Commission from Enron, during the company’s investigation for corruption and accounting fraud. Documents had to be scored according to responsiveness to a set of topics that were meant to represent the kinds of requests for production council might encounter in an actual case. Helioid’s approach made use of relevance feedback, returning an initial set of results and using a subset of the results, deemed relevant to what the hypothetical user is looking for, to expand the original query, and use the query expansion to reorder and personalize the results to the user’s needs. In a real legal case, this kind of feedback-reenforced personalization might reduce time and expenses associated with the review process, which is pretty sweet. Take a look at our submission and the result overviews.