Explain the historical trends in deep learning
Answer:-
Evolution Through Waves and Names:
- First Wave (1940s-1960s): Known as “cybernetics”
- Early models like McCulloch-Pitts Neuron and perceptron emerged
- Focused on computational models of biological learning
- Second Wave (1980s-1990s): Known as “connectionism” or “parallel distributed processing”
- Emerged in cognitive science context
- Introduced key concepts like distributed representation
- Third Wave (2006-present): Known as “deep learning”
- Began with breakthrough in training deep belief networks
- Currently focuses on supervised learning and leveraging large datasets
Growth in Dataset Sizes:
- Early Datasets (1900s-1980s): Small, manually compiled datasets like synthetic datasets and low-resolution images.
- 1990s-2000s: Datasets like MNIST (tens of thousands of examples) and CIFAR-10 improved learning tasks.
- 2010s-Present: Datasets like ImageNet (millions of labeled examples) revolutionized the field, allowing deep learning to achieve human-level performance. Supervised learning generally requires around 5,000 labeled examples per category, with human-level accuracy often needing at least 10 million labeled examples
Increase in Model Size:
- Neural networks have doubled in size roughly every 2.4 years.
- Growth is driven by advancements in hardware, such as GPUs and distributed computing, and access to larger memory.
- Modern networks, though much larger, remain smaller than biological systems like mammalian brains.
Improved Accuracy and Applications:
- Deep learning has evolved from solving basic tasks like recognizing individual objects in small images to handling diverse and complex real-world applications.Applications include:
- Image and speech recognition.
- Traffic sign classification.
- Reinforcement learning (e.g., playing Atari games).
- Machine translation.
- Image and speech recognition.
- Traffic sign classification.
- Reinforcement learning (e.g., playing Atari games at human-level performance).
- Machine translation and natural language processing.
- Deep learning is also widely used in scientific fields like drug discovery, particle physics, and medical imaging