Projectdetails:

Runtime: 01. April 2021- 31. März 2024

Projectpartners:

Laboratoire d’Informatique de Grenoble, Information Modeling and Retrieval research group (MRIM)

Industrial Partner:

Qwant

Funded by:

When evaluating a search engine, you need to test it under specific conditions—this includes picking certain ways of measuring performance, datasets and evaluation metrics. But the choice of these testing conditions is often made without a transparent reason. Also, no one usually measures what happens if you change these conditions. This is where the Kodicare-project comes in.

Projectgoals: KoDiCare addresses a long-standing but ever more pressing problem in information systems: The evaluation of search effectiveness in a reliable, cost-efficient, and reproducible manner in real‐world scenarios. The challenges the field of Information Retrieval (IR) Evaluation has faced over is decades of existence are: The heuristic nature of test collection creation and the consequent unpredictability of IR system performance when the test collection changes. This has led to a lack of adoption of IR Evaluation best‐practice in the industry.

Projectdescription:In KoDicare we measure the impact of changes to an IR evaluation environment (test collection + IR system) objectively by quantifying „knowledge deltas“, which describe a quantifiable understanding of differences between datasets, documents or test collections.
A controlled environment quantifying „knowledge deltas“ will support continuous evaluation and enable the provisions of explanations for system engineers.
The theoretical results will be confronted to real cases defined by a French company that deploys a web search engine (Qwant).

Projectdetails:

Runtime: 01. April 2021- 31. März 2024

Projectpartners:

Laboratoire d’Informatique de Grenoble, Information Modeling and Retrieval research group (MRIM)

Industrial Partner:

Qwant

Funded by:

When evaluating a search engine, you need to test it under specific conditions—this includes picking certain ways of measuring performance, datasets and evaluation metrics. But the choice of these testing conditions is often made without a transparent reason. Also, no one usually measures what happens if you change these conditions. This is where the Kodicare-project comes in.

Projectgoals: KoDiCare addresses a long-standing but ever more pressing problem in information systems: The evaluation of search effectiveness in a reliable, cost-efficient, and reproducible manner in real‐world scenarios. The challenges the field of Information Retrieval (IR) Evaluation has faced over is decades of existence are: The heuristic nature of test collection creation and the consequent unpredictability of IR system performance when the test collection changes. This has led to a lack of adoption of IR Evaluation best‐practice in the industry.

Projectdescription:In KoDicare we measure the impact of changes to an IR evaluation environment (test collection + IR system) objectively by quantifying „knowledge deltas“, which describe a quantifiable understanding of differences between datasets, documents or test collections.
A controlled environment quantifying „knowledge deltas“ will support continuous evaluation and enable the provisions of explanations for system engineers.
The theoretical results will be confronted to real cases defined by a French company that deploys a web search engine (Qwant).