Older news

Loose actual project timeline

These were the CLEF-IP 2013 Updates

CLEF-IP Workshop

September, 2013
Our workshop will take place at the CLEF conference in Valencia. See you there!

Evaluation Results

June 1st, 2013
Today we will release the evaluation results.

Topics for Claims to Passages task available

April 19, 2013
FINALLY! The set of test topics is done. It has been non-trivial to extract the sets of claims.
Download here the set of test 50(EN)+50(DE)+49(FR) topics. You can find them on the task description page as well.

The submission deadline for this task only has moved to May 10th!

Topics for Flowchart Recognition available

April 10, 2013
We're working on the set of test topics for the 'Claims to Passages' task. Bear with us a few more days.

April 2, 2013
The topics for the Flowchart recognition task are now available online.

Training data for 'Claims to Passages' task, Registration

March 13, 2013
We have done some updates to these pages for the 'Claims to Passages' task. The training data is now available here. Please look at the added 'tfam-docs' content to the topics on the task's page

We are closer to having the test topics, but generating their qrels will be time-consuming. Anyone wishing to help drop us an e-mail!

To register to our Lab please fill in this form on the CLEF 2013 page.

Choosing topics

February, 2013
We're in the process of choosing topics and creating relevance assessments. Meanwhile we've updated the track's time line. Do note that, compared to the previous years, the deadlines are moved with months forward.

CLEF-IP 2013 Tasks

December 7th, 2012
The summary of the tasks in 2013 are now available.

News on CLEF-IP 2013

November, 2012
Our proposal for the next CLEF-IP 2013 lab has been accepted. There'll be three tasks organized, stay tuned for updates.


October 4, 2012
The CLEF-IP collections are extracts of the MAREC data collection for which the following license is now valid:

Creative Commons License MAREC by IRF is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. Permissions beyond the scope of this license may be available at mailto:marec@fandan.net.
The CLEF-IP download pages are now freely acccessible. Please let us know if you find any broken links.

Workshop programme

September 12, 2012
We have finalized the CLEF-IP workshop programme at CLEF 2012
The evaluation results for the Flowchart task take way longer than we have expected. We will keep you posted about them.

The workshop notes for all CLEF labs are available on the CLEF 2012 website.

Evaluation results

July 31, 2012
The evaluation results for the Claims To Passage Task are ready!
The evaluation results for the Flowchart and Chemical Recognition Tasks will be published by the end of this week.

June 27, 2012
Details on the main measures in the claims to passage task and flowchart task.
The submission deadline is July 7th.

June 23, 2012
Chemical structure recognition task test data is now available.

July 9, 2012
Several requests from the participants made us delay the submission deadline to July 11, midnight.

June 29, 2012
We are setting up the details about the submission systems. Please send us your team identifier if you don't already have a DIRECT account.
We've created a page with FAQs about submissions here and updated the workshop page.
The CLEF 2012 early bird registration deadline is July 7th.

June 14, 2012
The CLEF 2012 early registration deadline is June 30.
The pilot task of the CLEF-IP 2012 lab is (pseudo)cancelled.

Please ignore the <citations> in the <bibliographic-data> section of the topic documents (Claims to Passage task). You can freely use the citations occurring in the <description> or <abstract> sections of the documents.

Submission System

June 5, 2012
Experiments to the CLEF-IP 2012 lab will be uploaded using the DIRECT system. Access credentials will be given later this month.

Test data available

June 5, 2012
It really was no easy matter, but we finally have the topic sets ready!
You can download the topic set for the Claims To Passage Task here.
The test set for the Flowchart Recognition Task can be downloaded here.
The topic set for the Chemical Structure Recognition Task are also partially available. Namely, 30 patents are available, from which you have to extract bounding boxes. We are not filtering through the chemicals to provide you a good set to run your recognition algorithms on.
The submission deadline is July 7th.

We have also created a page with information on patent documents. Look here to understand the difference between *A1.xml, *A2.xml, *B1.xml, etc. files. Use this together with the clarifications page for the claims-to-passage task to understand it better.

When looking at the topics in the training set, you surely have noticed that all XPaths are relative to A level documents (i.e. application document). The same is true for the topics in the test set. This is due to the fact that search reports allways refer to application documents as relevant citations. Therefore, all results will be from "A" documents only.

Updates on the Claims to Passages Task

May 9, 2012
We have found some errors in the training data qrels. Please download them again.
Also, we have written down some clarifications about this task, if you have further questions please send them to us.

May 4, 2012
The training data for the Claims to Passage Task is available together with a description of the task.

April 29, 2012
We are finalizing the set of training data. We will make it available in the following days - together with specific guidelines.

Flowchart training data available

You will find some detailed information about the task involving flowcharts here.
The test data will be made available beginning of June.

Data partially available

The corpus of patent documents is available to download.
Note that CLEF-IP 2012 uses the same patent documents as CLEF-IP 2011.

We are working on creating the training and the test topic sets, as well as the guidelines.
Some trailer-like information: for the passage retrieval we plan on involving XPath in the expected results. We are also very close to finalizing the textual representation of the flow charts, representation later used in the evaluation of the task.

Registration is open!

March 1st, 2012
See here how to register to CLEF-IP 2012