We helped a large consulting company to automatically detect requirements in their tender process
Our customer is a large consulting company that acquires many of its projects through public procurements. They create almost 30,000 tenders every year and have a lot to win on more effective handling. It’s also common that employees who work on bids forget to fulfil specific requirements or miss legal details, which leads to certain rejection.
In this project, our task was to develop machine learning algorithms that identify requirements and risks in the legal sections of project descriptions and reduce the number of mistakes made during the tender process.
The main challenge was that we didn’t have any data for training. There was a process in place, but no systematic storage of data good enough for training an algorithm.
To create training data for our algorithms, we developed an application called Lably to speed up the annotation process. You upload the data to annotate and start working. In parallel, we automatically train a machine learning algorithm as new annotations appear. The algorithm prioritizes which data point to annotate next based on certainty.
We uploaded sentences and paragraphs from relevant documents that our customer started annotating. Once the algorithm reached the desired performance (after around 5,000 data points), we went straight to implementation. Here's how the algorithm improves over time with accuracy on the y-axis, and number of data points on the x-axis.
To make the algorithm accessible to our customer´s employees, we create a simple web application. User upload their PDFs to the algorithm, which returns a list of requirements and potentially risky paragraphs.