Sometimes we need to quickly analyze or validate our structured data. Generally, people tend to use pandas since it seems easy and visually good. The problem with pandas is that it can be super slow if you have large data.
We have another alternative which is Linux commands. Linux commands run super fast (even on huge data) and you don't even need an IDE/Notebook to run that. You can just execute them in your terminal.
I am going to use the famous titanic data in this tutorial, as an example. …
Training a Neural Net on permutation invariant data is difficult. And a jigsaw puzzle is one of those.
A function is a permutation invariant if its output does not change by changing the ordering of its input. Here is an example below.
1) f(x,y,z) = ax + by +cz2) f(x,y,z) = xyz
The output of the 1st function will change if we change the input order, but the output of the 2nd function will not change. So 2nd function is permutation invariant.
Weights of the neural net maps to specific input units. When the input changes, the output will change too. So in order to learn this symmetry, weights should be such as that final output is unchanged even after changing the input. …
I am going to explain a really basic but important topic, Axes and Dimensions. Many people find it quite confusing, especially using axis while applying a function on multi-dimensional data.
Axis or dimensions is a very generic concept. Whether you are handling data in Numpy, Pandas, TensorFlow, or another library, you have to encounter it frequently. And the concepts I am going to explain will be common across all these libraries.
In simple words, the Axis is something that represents the dimension of data. Let’s go through various examples to understand it at its core.
A Scalar is zero-dimensional data. It has no dimensions or axis. …
The Loss Function is one of the important components of Neural Network. Loss is nothing but a prediction error of Neural Net. And the method to calculate the loss is called Loss Function.
Loss is used to calculate the gradients for the neural net. And gradients are used to update the weights. This is how a Neural Net is trained.
Keras has many inbuilt loss functions, which I have covered in one of my previous blog. These loss functions are enough for many typical Machine Learning tasks such as Classification and Regression.
But there might be some tasks where we need to implement a custom loss function, which I will be covering in this Blog. …
You might get into a situation where you need to label your Images for training. Or you already did. One way of labeling is to manually look at each image and write down its label in a file, which is a painful process.
In this blog post, I am going to share how you can make this process fast and easy. Just by writing a few lines of code in Jupyter Notebook. Let’s see the demo first.
I have some cat and dog Images in my
Image directory. I want to label a cat as
0 and dog as
1. I just have to call a function by passing the Image directory. …
I was really interested in an idea, which was to read the time from Analog Clock Images using Neural Nets. To do this task, I needed a dataset containing clock images, but there was not any dataset available on the web. One other way was to download the clock images from the web and manually label them. Which is a quite time-consuming process.
Finally, I decided to write a python script that generates animated clock images and their respective labels. The following are the few samples of clocks images generated by the script.
These images are not as realistic as real-world clock Images but it would be really exciting to see if a Neural Net can be trained on these images as well. …
I am not going to write about complex maths and algorithms behind the Neural Network. Rather I am going to build your intuition on how a neural network works. I am going to do so by creating the smallest neural network and train it to do a simple task.
Intuitions are not to be ignored John. They represent data processed too fast for the conscious mind to comprehend - Sherlock Holmes
A neural network is a collection of weights. We can train a neural network on a set of input and output(target or label). The weights inside the neural net interact with the input and produce an output. As the network is trained, the weights are updated such that it tries to match the output with the target value. In short neural net learns the mapping between input and output. …
Originally Python was not designed for numeric computation. As people started using python for various tasks, the need for fast numeric computation arose. And the Numpy was created by a group of people in 2005 to address this challenge.
Today in the era of Artificial Intelligence, it would not have been possible to train Machine Learning algorithms without a fast numeric library such as Numpy.
The following are the main reasons behind the fast speed of Numpy.
People have been using reinforcement learning to solve many exciting tasks. Whether it be as simple as atari games or as complex as the game of Go and Dota. Reinforcement learning not just have been able to solve the tasks but achieves superhuman performance.
In this blog, we are not just going to solve another reinforcement learning environment but going to create one from scratch.
To those, who are not familiar with reinforcement learning and wondering what an environment is, let me give a brief intro. …
Keras contains 10 pretrained models for image classification which are trained on Imagenet data. Imagenet is a large collection of image data containing 1000 categories of images. These pretrained models are capable of classifying any image that falls into these 1000 categories of images.
This guide will cover the following concepts.
In a simplified way, the Image classification model looks like the following.
Which contains convolutional layers followed by fully connected layers. …