Lightning Talk: Ball Tracking and Detection in Soccer Videos - Comparison of... Maciej Szymkowski

In this lightning talk, Mati Sumkowski from Future Processing discusses his research focused on ball tracking and detection in soccer videos. He begins by explaining his motivation rooted in a lifelong passion for football, despite his lack of athletic ability. The presentation covers the use of the YOLO V11 model for initial experiments, highlighting its efficiency and suitability for detecting soccer balls in various scenarios. Sumkowski also explores the application of vision language models, including Gemini, Qwen, and InternVL, to enhance detection capabilities. He shares insights on the dataset used, which consisted of 12,000 images, and reflects on the varying performance of the models, ultimately concluding that Gemini provided the best detection results. The talk wraps up with plans for future work, including experimenting with newer models and continued exploration of object detection within soccer match footage.

Good afternoon, everyone. I'm really glad to see all of you during my short lightning talk. So, my name is Mati Sumkowski and I'm representative of future processing. Today, I would We are a company from Poland. Today, I would like to say a couple of words about my little research. Of course, this is a like really initial results, but anyway, there are some interesting outcomes I would like

to share with you. Maybe I will start why I started this sort of topic. Well, from the from early childhood, I was mad about football. I also thought about being a soccer player, soccer footballer, but well, my abilities to play football are not really well not really great. If I kick ball to the left, it it moves to the right. So, not at all. I'm not a

good footballer. But, I still love to watch football, watch soccer matches. Even today, PSG is is playing against Liverpool. So, one match one match more. But, anyway, what I would like to say is that in my mind, I had how can I also be included in the this huge world of football? So, it is why I also tried to provide some my own solutions. And it is

why the first of them was related to ball tracking and detection in soccer videos. So, briefly, I will try to say a couple of words about the database, then about the first approach, simple approach with YOLO V11. But, also, I tried to evaluate vision language models. From my point of view, they are also kind of an interesting solution. So, well, well, I will try also to say

whether they are whether we can use them for object detection in the sky in this situation of a ball. And at the very end, conclusions and future work, so like a typical academical presentation. So, let's start with the database. At the very beginning, I thought about SoccerNet, but well, in fact, it is a really huge dataset, around of 10 GB for a one data set. So well,

my resources were not enough to train such a huge model. For now I I hope I keep my fingers crossed that it will be better in the coming months. So I started with a much smaller data set called ball detection computer vision model. It is available on Roboflow platform, but it is also interesting because it shows different situations from the training ground. Mostly there are some scenes

where the the ball is only visible, but also people are trying to do some activities with the ball. What is more, there is one more interesting point is that there are scenes where there are plenty of balls. In a moment I will show you some examples. But anyway, what was also important for me is that all the samples have their ground truth versions. So it is perfect

that they were marked. Of course, one of our colleagues here in the in the open space was also showing his own platform. I would like to also check it in a in a couple of days. But moving back, the data set consisted of around of 12,000 images for training, 44 images for testing and 41 for validation. Not really huge, but for the initial for a kind of

a toy model, it was enough to to check. So what kind of examples we can see? So on the left hand side we can see like a training ground and here we can see four balls and also on the right hand side we see a man who is doing some kind of an activities with So I started with YOLOv11. So one of the models that are that

is really well known. In fact, in this case it is it was selected not only due to high efficiency, accuracy and how well it is working, but also I keep in mind that in the further versions of this work I would like to also deploy the models within a single board computers. So, YOLO is also able to to be integrated within this this sort of this sort

of solution. And what is more, for fine-tuning, we did not this model do do not require any resource heavy environment. In fact, my gaming PC was enough to to finish the process in a really short time. Of course, this I have an video RTX 4090, so quite quite good quite nice graphic card. But also I from the very beginning I set the fine-tuning process for 250 epochs,

but the condition of early stopping that is implemented within YOLO was was reached and it was finished after 155. Well, if we'll take a look at different different schemes, we we can see that the metrics looks really nice. Of course, high precision, high high recall, so the the source of metrics we would like to see. So, everything looks fine. And if we'll take a look on the

validation set, well, it's quite okay. So, we have here the soccer balls, we the also the certainty of the model, so we can say that okay, it works. but it doesn't. Simply because if we'll sorry, if we'll take a look on this the this image, we can see that the model setup only showed showed us three balls. But what about this? Neither of them was observed. And

also when I checked the results about related to the detection, it didn't show any sign of this. So, even no low low level certainty was was here. So, it is a huge problem. So, what I would like to also check was related to visual vision language models and whether they can work. Of course, I will start with a little drawbacks because we have here high VRAM demand

in the case of resources. This inference time can be much longer than in the case of the specialized pipeline and single board computers probably will not be able to run this models on these devices. Of course, today I also heard about Gemma 4, so probably it will be one of the models I would like to also check. But anyway, in this part of the experiments, I gave

a chance to free VLM, so Gemini free, Qwen and InternVL. And to be fair, all of them got the same the same prompt, so you are a specialist in analysis of soccer images. Your task is to mark all balls in the image. There can be more than one ball. As the result of your work, return information how many balls are in the picture, their position their positions

and your certainty about the detection results for each of them. And when I asked Gemini, wow, the result was impressive. All the balls were were observed. So, 11 balls, also information about that the this balls from number 4 to 11 are on the goal line and that was Also, the positions of the rest of the balls, so these three that are closer to us were also marked

correctly. When I asked Qwen, it was okay because it also marked these three balls that are close to us. Also, there is information that probably there is a fourth ball. I suppose that it is the one that is close to the goal, so okay. But anyway, this is not the result I would like to see. This is not the 11 balls, but only four, but it's better

than YOLO in this case. Anyway, what I didn't mention before, Qwen and Intern were deployed within my own resources and while the Gemini was run within the while And when it comes to the intern Well, it also marked three balls and nothing more. Unfortunately, there was no information about any additional positions only three balls that were that were marked also by by Yolo and all the rest

of them of the models. So, moving moving on to the conclusions. As I said, this is a this was a really initial experiment some start of the journey with the with the images of from soccer matches. Previously, I was working with medical images. So, it is a not slightly, but really different area. But anyway, it made me really fun because I I like soccer. I like football.

So, it is why I started this. And in the case of this, I need to say that Gemini was the best one. It was the model that reached the the highest certainty and the highest precision. In the case of Yolo Q and an intern, they well, their results were not the best ones. Still, not all the balls were When it comes to the next stages because the

next stages will also be in a in a of days, I would like to also continue with the newest Yolo. So, Yolo V26, but also RFDITR. This is the one of the models I I read about in the paper from Nvidia and also continue working with with LLMs and VLMs to to observe this information from from football matches. So, thank you very much for your attention. If

you would like to reach me, please feel free to connect on LinkedIn. It will be a pleasure also to discuss some potential joint projects. So, thank you very much for your time.