Dairy Farm Analytics Company
The Challenge
Annotating Thousands of Hours of Video Daily
The Client relied on a team of remote workers to annotate video recordings received from client dairy farms. The workers would observe their assigned videos and make note of any deviations from the milking protocol that could lead to a loss in revenue for the farm. This meant looking out for workers using dirty towels to wipe cows, workers behaving aggressively or wasting time, inefficient use of resources such as water hoses, empty milking stalls, and up to twenty other deviations, all of which could lead to a loss in revenue for the farm if left unchecked.
With close to 60 customer farms with multiple milking parlors and thousands of cows each, the workload was immense. Thousands of hours of video needed to be annotated every day, and the remote team did not have the speed or accuracy required to get through all of it.
The Client needed a cost-effective and reliable solution to replace the slow and expensive remote workers, who often had power outages or other issues preventing them from working, leaving the Client struggling to meet demand. They needed a centralized team with its own management infrastructure that could be trusted to deliver fast, accurate annotations, and could rapidly scale up when needed. This is where HiredSupport came in.
Taking over the full gamut of video annotation from the Client’s established team was not something that could be done overnight. We needed to prove that:
- We could do a significantly better job than the remote staff, both in terms of annotation speed and quality,
- We would be able to keep pace with the client’s expansion plans, as their projections had them at double the number of farms after one year, and
- We could do it all at a more cost-effective rate than the remote workers.
The Solution
Building Strong Foundations by Starting Small, then Scaling Rapidly
Based on our experience running large teams in customer service and transcription projects, we knew that with undertakings of this size and scope having the right founding members is critical. This is why we tapped one of our best and longest-serving staff members to be one of the founding members. Not only did this give him the chance to grow into a leadership role, but it also got us a thoroughly vetted person at the head of the project who we could trust to lead it through its critical formative stages.
Our founding team of four went through a rigorous four-week training regimen where they learned how to spot and record over two dozen issue types across three different styles of milking parlors. At the end of the training period, they had to face a test where they were tasked with spotting multiple instances of 26 distinct issues over the course of a 2-hour video. All four members qualified on their first attempt and started work on real data the very next day.
Right out of the gate, we faced three challenges that we needed to overcome before we could even think about expanding. These were:
- Correctly identifying issues that needed to be recorded, while ignoring the ones that fell within acceptable limits or were not applicable to that particular farm.
- Walking the tightrope between playback speed and observation clarity. Normal playback speeds were out of the question except for parts with high activity. We needed to get comfortable observing at a minimum of 4x speed without missing issues.
- Keeping annotation speed and annotation quality in the green. Each week the Client would use its in-house QA team to review a subset of our annotated videos. We needed to keep our speed and observed issue count within 25% of the QA to meet their expectations.
We brought on a second batch of four after the first had completed a month of work, and it was during this batch’s training and onboarding that our path to addressing these problems became clear:
- First, we built out what we called an ‘encyclopedia’ – a document with exhaustive information on each of the 26 issues, detailing how to spot them, record them, and to which farms they applied. This encyclopedia was constantly updated as new farms were brought on or policies were changed based on feedback from client farms.
- We learned to use hotkeys to complete repetitive actions such as pausing, speeding up, slowing down, frame-by-frame analysis, and timestamp recording as quickly as possible. We learned to predict during which milking activities certain issues were more likely to occur, slowing down in those parts, speeding up the rest. For people who were really struggling with speed, additional coaching was provided by the fastest team member, leading to marked improvement in their performance
- We gradually ramped up the workload for the new batch. While prioritizing quality, we encouraged them to ramp up the speed over the course of a month till they were able to annotate 8 hours of video in a standard workday while correctly identifying at least 85% of the issues present in each video.
Now that we had a solid foundation, we felt ready to ramp things up. Our senior team member assumed the mantle of Team Lead but still continued to annotate videos for 20 hours a week so he could stay in practice. With the Client’s blessing, we took training of subsequent batches in-house, managing to halve the duration to two weeks, leading to a much quicker ramp-up time for the annotators. We tripled the team’s size in the next two months, and at the Client’s behest, established a new team to provide daily performance reports to larger farms who needed to track metrics other than deviations from milking protocol.
The Results
60+ Annotators in 6 Months | 90%+ Quality | Quicker Than QA
The first team lead became the Project Manager and successfully grew the team to 60-strong by the 6-month mark, all the while maintaining overall quality at an impressive 90%, with annotation speed within 5-10% of the Client’s best QA people. The performance reporting department grew to its own 20-member team with its own team lead.
It was at this stage that we faced our biggest challenge yet. Our Client announced they intended to onboard an additional 22 farms with 132,000 cows between them, all in the short span of one month. With our prevailing ratio of 1.3 – 1.4 annotators per farm, we forecast the need for 30 additional annotators. For the ill-prepared, such a scenario can spell doom, as scaling this much this fast tends to expose weaknesses in recruitment, training, and management practices that could have otherwise remained hidden.
Thanks to our deliberate efforts to establish a strong foundation in the beginning, we were able to rise to the challenge, bringing on the first batch of 16 on the 4th of the month and completing their training on the 20th, with the second batch of 14 starting on the 18th and qualifying on the 30th. We effectively trained and onboarded 30 data annotators ready to start annotating live videos in one month.
With the new team members in play, we comfortably met the surge in demand without causing any backlog elsewhere, bolstering our Client’s reputation with their customers and investors. Our ability to scale cemented our commitment to take our joint partnership to the next level. Our Client has fully put their trust in HiredSupport for the entirety of their annotation requirements and we continue to deliver and grow the team every month.
