Advanced Machine Learning at the National Research University Higher School of Economics

I am currently studying advanced machine learning at the National Research University Higher School of Economics, which is a top Russian university for economics and ranks among the most significant universities for economics in Eastern Europe and Eurasia. The school is rigorous and demanding.

As an exercise on convolutional networks, we are tasked with implementing cifar_10 according to their specifications. Any deviation is not permitted. In this dataset, you have 50,000 images divided into 10 categories such as aeroplanes, cars, birds, cats, horses… It's amusing to see how the trained network makes mistakes.

This is a network that has been trained for just a few minutes, and it correctly classifies about 70% of the images so far. However, it is already evident that it often confuses horses with dogs. 🙂 Dogs are also confused with cats. When the network misclassifies a bird, it most frequently guesses dog, cat, or aeroplane. Isn't that wonderfully human-like?

This is a network that has been trained for just a few minutes, and it correctly classifies about 70% of the images so far. However, it is already evident that it often confuses horses with dogs. 🙂 Dogs are also confused with cats. When the network misclassifies a bird, it most frequently guesses dog, cat, or aeroplane. Isn't that magical?

A stronger network that I let train for over an hour. On random examples, it confuses horses with deer and boats with aeroplanes. I have a feeling that many people would make fewer mistakes.

Finally. It took me quite a while to display what convolutional filters (kernels) look like. There are a total of 4 convolutional layers here, with 1 being the least and 4 being the most abstract.

The highest possible level of abstraction, that is, the visual representation of the last dense layer. Plato would be pleased, as this is the pure form of ideas: filter1=aeroplane, filter2=car, filter3=bird, filter4=cat, filter5=deer, filter6=dog, filter7=frog, filter8=horse, filter9=boat, filter10=truck.