A lot of you (hopefully) were looking forward to our initial release of our patent search platform, "Dorothy," in May. I was too. However, we decided to pump the breaks, slow our roll, and do more testing before letting customers or potential customers onto the platform.

This goes against "conventional" wisdom, which pushes tech entrepreneurs to get their MVP (Minimum Viable Product) in front of customers quickly. In a world of video game chat and emojis, this makes complete sense. Customers will buy the product or not and complain or glorify it on social media for everyone to see. Here, it does not.

We are scientists. Our customers, patent attorneys and agents, are scientists too. I hear the skepticism in your voices, when I tell you about sorting by relevance and finding the most important references in the first 25 results. Heck, I spent dang near every minute of every day of the last year working to bring Dorothy to life. I REALLY want to trust the results. I don't. Yet.

What will make me feel better about our platform? Data!

We have set out to quantify the quality of search results. We have designed searches for issued patents in each CPC section. After identifying the novel feature of the independent claims using the patents file history, we developed a detailed Boolean query based on the novel feature. This Boolean query was entered into Google Patent and a commercial patent search platform. The query was tested by looking for the patent on which the search was based in the first 5 results returned by each platform. When a query met these criteria, the results from both Google Patent and our commercial platform were aggregated and compared.

We are looking at every single returned result from each platform to identify the 20 most relevant references for each search query. In other words, we are feeling the pain we are trying to eliminate. Once we have the 20 best references for these searches, we can compare the results from any search platform, or outsourced search, based on whether these, or related, references appear in the first 25, first 50, first 100, or not at all in that platforms list of returned results.

We can use the same protocol to test Dorothy. We will feed the independent claim associated with the search into the "Invention Disclosure" text box, and the Boolean query into the "Novel Feature" box. Returned results will be scored based on whether the 20 best references appear in the first 25, first 50, first 100, or not at all. In this way, we can compare Dorothy to other platforms across different technologies defined by the CPC sections.

I look forward to sharing the results of our experiment with you, particularly if Dorothy performs better that our baseline platforms. Even if she doesn't initially, we're establishing a benchmark that we can work towards based on data rather than "likes" on the App Store website, and hopefully building trust with our customers.

BTW, I'm happy to share our test Boolean queries, if you'd like us to evaluate the platform you are currently using as long as you share the results.