1. ResNet50 vs ViT ::: Who Handles Chaos Better?

This research focuses on a comparative analysis between two powerful deep learning architectures, ResNet50 and the Vision Transformer (ViT), particularly when they are subjected to datasets with varying levels of intraclass variance.

The experiment was designed to highlight how each model responds differently depending on the complexity and variability present in the training and testing data. To establish a baseline, both models were first evaluated on simpler, well-known datasets such as MNIST and Omniglot, which are characterized by low intraclass variance and relatively straightforward visual patterns.

In these controlled conditions, ResNet50 demonstrated outstanding performance, achieving an accuracy of 99%, which underscores its exceptional capability in recognizing structured and repetitive patterns with high precision. The ViT model, while slightly behind at 96%, still performed admirably, though the gap in performance at this stage already hints at the fundamental architectural differences between the two.

ResNet50, being a convolutional neural network, is inherently designed to extract local spatial features through its deep residual connections, making it particularly well-suited for datasets where patterns are consistent and predictable. The ViT, on the other hand, leverages a self-attention mechanism that processes image patches globally, which may not offer the same advantage when dealing with simple, low-variance datasets but begins to shine when the complexity increases.

When the experiment transitioned to datasets with high intraclass variance, where samples belonging to the same class can look drastically different from one another, the results told a very different story. ResNet50's accuracy dropped sharply to 56%, revealing a significant limitation in its ability to generalize across highly variable data distributions.

The model's reliance on local feature extraction becomes a bottleneck when the visual characteristics of the same class differ substantially. In stark contrast, the ViT model maintained a considerably higher accuracy of 77%, demonstrating its superior ability to capture global context and long-range dependencies within an image.

This contextual understanding allows the ViT to better adapt to scenarios where the relationship between different parts of an image is more informative than local textures or edges alone. The research ultimately emphasizes that neither model is universally superior, and each has its own domain of excellence, so the choice between ResNet50 and ViT should be guided by the nature of the dataset and the specific demands of the task at hand.