AI Perk Identification setbacks – Everything is windows!

I recently have been trying to train an AI model that will be table to ingest a screenshot of the post-game dead by daylight scoreboard and identify all of the perks that are being used. Here is an example of the training data I generated (I played a bunch of games and took screenshots, which I loaded into Roboflow and generated the data as shown below)

With each of the perks on the 55 screenshots I uploaded tagged with what those perks were, I’ve been running it through yolov7. I’m about 500 epochs through my training data and the results aren’t super promising. The only confidence score of note so far is for the perk “Windows” — the most common perk seen by far. Unfortunately, it sees the highest confidence score on all of them as windows.

Clearly I’m doing something wrong on the training, but before I make a change I’m going to allow the training to continue through the number of epochs I had scheduled and test once m ore.

You can see that the perks that are *actually* Windows are showing 0.44 confidence score in my preliminary testing, where the ones that are *not* windows are less than 0.1. So with the current model, if I were only trying to evaluate for the occurrences of the perk “Windows” i could set the confidence score minimum to something like 0.3 and it would be relatively accurate, but that’s not what I was aiming for.

I know my sample set isn’t good enough for *all* of the perks, but I expected the most common ones (not just windows) to come through.

I’ve got a sneaking suspicion that I’ll need to have two passes here.
Pass 1: Bounding boxes on “Perks” which I will use the X Y coordinates + height and width to crop and resize each one to 640×640
Pass 2: Push each of the cropped and resized perks through a second model which will identify them individually.

I’ll call this “Plan B” because that’s going to require quite a lot of additional work, so in the meantime i’m going to run this model through training a long while and work on another project while it does. I need to get back to my locator node project anyways.