Tokenization method for LLaMA, T5, XLNet — Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates [ACL 2018] [github] Publish month(arxiv): 2018.08 Code: [Colab] Tags: nlp, tokenization ✰ Best viewed in [Notion] Overview Sentencepiece is both an improved tokenization algorithm and implementation by the researchers at Google. The terms below seem overwhelming, but they are very…

Is multithreading or multiprocessing faster than another? — ✰ Best viewed in [Notion] Overview In Python, multiprocessing and multithreading are primarily important for improved performance. Both multiprocessing and multithreading help maximize the utilization of a system’s CPU and other resources. By distributing tasks across multiple threads or processes, they enable parallel execution, which can lead to significant performance improvements…

Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates [ACL 2018] Publish month(arxiv): 2018.04 Tags: nlp, tokenization ✰ Best viewed in [Notion] Subword Regularization: Improving Neural Network Translation Models with Multiple Subword… A sentence can be represented in multiple subword sequences even with the same vocabulary. Previous methods don't…deep-learning.notion.site

Faster R-CNN is still an actively cited benchmark for comparing object detection performance of modern network architectures. In this post, we will discuss the insights and contributions of the papers. Introduction — Why do we even care in 2022? Faster R-CNN was a huge breakthrough in computer vision because it enabled real-time object detection and enabled end-to-end training of…

CNNs don’t always have to progressively decrease resolution. A revolutionary idea that might shape the next-gen architectures for Computer Vision. — TL;DR: Transformers(NLP): Isotropic architecture+Self-attention ViT: Transformers(NLP)+Patch representation CNNs: Pyramid architecture(decreasing resolution)+Convolution ConvMixer: Isotropic architecture+Patch representation+Convolution Transformers are represented by the extensive use of attention and their isotropic architecture, which repeats the same block multiple times. They demonstrated amazing performance for many problems, especially in natural language processing. However, the…

In-depth review on Basic, Transposed, Dilated, Separable, Depthwise, and Pointwise convolutions and their applications. — Recent research on CNN architectures include so much different variants of convolutions that confused me while reading these papers. I thought it would be worth it to go through the precise definitions, effects, and use cases(in computer vision & deep-learning) of some of the more popular convolution variants. …

Deep-learning is used everywhere, but they often overfit the training data and are not generalizable to unseen samples. Various regularization techniques prevent such overfitting. Dropout is one of the most popular and powerful regularization techniques, used regardless of network architecture and type of task. …

VGG is considered one of the cornerstones of deep CNNs. The authors of VGG provided widely accepted principles for designing CNNs. In particular, they suggest that multiple 3 × 3 convolutions are more efficient than using larger kernel sizes. However, the VGG architecture, composed of nothing but successive stacks of…

Object detection is a complex problem in computer vision that involves localizing and classifying multiple objects from a given image. It is one of the most practically important and widely researched problem in computer vision. There is a wide variety of approaches to object detection including deep-learning based methods. These…