Significant DBD-AI Modeling Progress (YOLOv7)

I mentioned in a previous post that I had some issues with the results of my yolov7 ai model which was supposed to identify perks based on the image in Dead by Daylight. They were all coming back with “Windows” as the perk for every single one. I’ve now found a way to increase that *significantly*!

There were a few parts to this. The first was to add a lot more data to the training set, and allow the training set to run through for a lot more epochs. I ended up with about 1500 images in the dataset (70% training, 20% validation, 10% test), and I need to get even more because I was only using screenshots from the actual games I played. That means the perks in the data set are only the ones commonly played (or uncommon ones I happened to run into).

Step 1: Create a new model that highlights each perk, but has them all as just “perk” – This will give me the x + y coordinates and height and width of every perk in a given screenshot. I took that information to cut each perk and save them into an array, then create a collage of just the perks. Why? This step allowed me to remove all false detections from around the screenshot that were not the perks. Ex: Players, names, title bar, etc. Removed all those false positives.

Step 2: I fed the collage into the newly trained model with the expanded dataset which gave me fairly accurate results. The few that are not accurate are either underrepresented in the dataset or aren’t in it at all, like killer perks. I’ve only trained survivor perks at this point.

Here is an example of the output collage with the labels on the perks. If you know the game, every one of those perks is labeled correctly. I’m pretty excited about that.

That said, it’s not always consistent. In the below example, there are some killer perks that are being labeled (small game, deliverance on line 1 are actually killer perks which have not been added to the perk model yet). Those will be solved once I add those to the model and train with those added in. Also, the second line has “sabo” listed which is a survivor perk, but that perk only had I think two instances in the 1500 images. I suspect that’ll be resolved when I expand the dataset further with more instances of the less popular perks, and run through more training.

I’ve managed to automate a lot of this process now. I’ve got a desktop application (python based) which takes a screenshot every second, and reads the top left corner for “Screenshot” as mentioned before. Now, in addition to identifying which killer is being faced, it also saves a copy of the screenshot to a network drive that lives on my modeling server.

The server also has a process I wrote running, which does the whole process I mentioned above automatically. It:

1. Monitors the share folder for new screenshots.
2. Pulls in the screenshot.
3. Runs the screenshot against the first AI model to identify each perk slot within the image
4. Cuts the perks out of the screenshot and creates a collage of them.
5. Runs the collage of perks through the second AI model which will identify what each perk actually is within the model.
6. Saves a copy of the collage (shown in the screenshots above) to a specific foldler
7. Adds a row for each in a sqlite database table which includes the ID, Perk name, timestamp and file path of the collage it was pulled from.

As I’ve developed it, I decided that it would be best if I could first cut each player out of the screenshot, so that I can store 3 separate tables. I’m now trying to make that happen, but early setback have been happening.

I really want the box sizes to be approximately the same, so I can easily do OCR on the top section of the box and remove any known character name (and any text ahead of it). I also am trying to train ig to identify which box has the gold background since that is the player who took the screenshot. I could run the perk detect model against just that section then, along with a model to detect if the result was an escape or not. With that information saved to the database, I’d be able to report on what perks have the highest probability of escape for *me specifically*, since every player’s playstyle is different.

The killer box has a red tint background to it. If I can get this model to properly identify what rows are Me, teammate, and killer, I could have the killer perks trained in a separate model as well instead of having to lump it into the same model as the survivor perks. I could have those take different paths on the server side code.

Longer term, I’d like to have a login system that will POST a screenshot and the playerID (account ID for my application for a given user) to my server, which will then set about processing the data and adding it to the database. With the data being on the server then, I could have many people install the application and it would collate the data for all survivors, while still having it flagged for one specific survivor.

Stay tuned for more updates on that.