TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications.
Before you continue reading let me clear WHY TF-1.14 when TF-2.2 is available? Its because you are bound to get stuck here and there, with importing libraries, getting all sort of weird error and the perk of using old model is you got many users stuck on the same problem as you and array of solutions, if one doesn’t work you get 99 others that might work, its tried and tested. And after you are comfortable with all ins and outs you can easily switch to a newer version at your pace.
TensorFlow is home to all AI/ML tools out there, tools here refer to predefined libraries and functions that will save tonnes of your time. The beauty of TensorFlow is the ability of the program to convert data to tensors and then process it in parallel thus using maximum resources it can to save time and produce optimum results. What can be the best example of tensors then image/ video processing where you have colossal data, this domain of AI is called Computer Vision.
The goal of this article is to help users to install TensorFlow in windows which will later be used to set up TensorFlow -model garden to run object detection on images and then videos.
Computer vision is a field of artificial intelligence that trains computers to interpret and understand the visual world. Using digital images from cameras and videos and deep learning models, machines can accurately identify and classify objects — and then react to what they “see.” I am sure if you have stumbled upon this article you are comfortable with some AI term (CNN, TF, CUDA)and don’t worry if you are not, I will be dropping down, links for almost every topic which will help you to explore it further. ex- A Gentle Introduction to Computer Vision.
MNIST, CIFAR-10, CIFAR-100, Cats and Dogs, Iris all these datasets are very good in understanding the basics, all the building blocks. Keep in mind we will not be designing network/ model architecture from scratch which can be done in Keras for MNIST, CIFAR instead we will be using a Pre-trained Network also popularly known as TRANSFER LEARNING(SSD-MobileNet_v2) and modifying it to our heart's content to full-fill our goal. SSD is the advancement of VGG, you can easily implement VGG in Keras using this nice tutorial.
But there is a difference between classification, detection on a single image vs classification, detection, tracking in series of images i.e video. Many factors join in when we are talking about running the same algorithms on video, such as FPS, Video resolution, type of input used (webcam, mp4, RTSP, UDP), GPU, CPU, memory allocation for GPU, etc.. here come the PRACTICAL ASPECT of the Custom object detection which will be laid out in detail further in this series of articles also we will see the limitations of pre-trained network and work our way around it, to some extent. Libraries such as Opencv, Pandas, NumPy, scipy will be used extensively.
The steps can be broadly divided into the following:
- Setting up ANACONDA-PYTHON 3.6 with TensorFlow-1.14-GPU
- Working with TensorFlow Model Garden
- Learning about Models, Running Inference on Images.
- Practical Approach towards implement Computer Vision i.e custom training on Vis-Drone dataset.
- Tracking and other Practical aspects of the same.
Setting Up Environment
The most tedious process is the environment setup and the selection of IDE to be used. Pycharm, Jupyter Notebook, and various others are very popular. I practice using SPYDER as it comes bundled with ANACONDA, for me seeing is believing, until unless I see the format of all the variables being used, I can't grasp their purpose also it helps exponentially when you are doing vector operations where dimensions come in handy.
NOTE: Various software/dependencies/versions are of utmost importance. It's a very delicate and vicious loop and can easily throw you in dismay, you may have faced it already and you may face it, again, and again, one size doesn’t fit all here. Be patient and try all logical solutions found in stack overflow, GitHub, and try to follow the same versions I will be provided later. Some key libraries to keep in mind according to importance:
- CUDA ToolKit(with Nvidia drivers)
- NVIDIA cuDNN
- Anaconda-python 3.6
- TensorFlow -1.14 (conda env)
- Opencv -4.2.0
What IDE to use?
SPYDER represents a very neat layout of all the variables and their respective data types, it helps to debug the code frame by frame and see exactly what went wrong and what was the possible reason.
Anaconda is an entire python universe properly segregated with multiple features to use and comes with, n number of isolated environments. There is no need to install python separately, all can be done with one executable file. If your goal is to only use TF for ML you can skip the theory net which explains the importance of GPU for image processing.
GPU (NVIDIA only) plays a major role in training and in inference (getting results after the model is trained), especially in the COMPUTER VISION field where the data is humongous, and when we are dealing with videos we see a humongous flow of dataset. An HD(High definition) image is size 1280x720 pixels, i.e width and height, every image can be broken into 3 fundamental channels of COLORS RED, GREEN, BLUE. now we have an array of 1280x720x3, every pixel has a brightness value in their specific color domain ranging from 0–256 (8-bit image, 2 in power of 8 = 256). So 1–8-bit image is an array of 1280x720x3 = 2764800 numbers ranging from 0–255. will look something along the line of:
This is one part of 1 channel among 3, different cameras have different rates of capturing frames the optimal is 30 FPS so an HD video will have 30 images (assuming color) of size 1280x720x3 so in total 82944000 numbers ranging from 0–255 that is all in one second. All this data at so much rate is way too much for CPU even if you are distributing the work across multiple cores, you can only have so many cores. Disclaimer: Do Not try to compute Video size in MegaBytes using the numbers calculated above cause it will sum up to about 1.5 GB per sec to run along the same figures would mean we need to have 8 Tb of HDD to store a 90 min HD movie. That’s preposterous, here comes VIDEO COMPRESSION to the rescue which reduces the size of the video by multiple folds, in order to learn more you can go on studying Laplacian and Gaussian pyramids.
So what can be done to process so many digits, using 1 core of CPU will definitely not be the solution, so we can deploy multiple CPU cores using the concept of parallel processing, still you will need a very high Core count in CPU? See the below Image to see the most expensive CPU available in commercial space.
The price for such rig will be detrimental to one's heart, so NVIDIA built a very user-friendly (not) library to be used with NVIDIA GPUs called CUDA.
Now, this CUDA can only be used with NVIDIA GPUs. Every GPU have a compute capability number, NOT every NVIDIA GPU is compatible with CUDA. Check here to find if your NVIDIA GPU is supported or not.
After getting all ducks in line, continue with the next steps. Till now, I have only given an overview of some important topics.
NVIDIA DRIVERS- Install this Before we begin with ANACONDA.select your respective GPU and download all the drivers, it may look it's for games but its a whole package and includes drivers such as CUPTI, which will help CUDA and cuDNN to install correctly.
F1-A) How to install TensorFlow with GPU support on Windows 10 with Anaconda
Anaconda archive lists the versions of ANACONDA’s for all the supported platform(not supported on jetson nanodevice as of 2020) Download and installs the executable file(700 Mb) for your windows (32/64 bit, python=3.6).F1-A) step will make sure you have TF-GPU installed on the system, the chances are it will not install the version we want but instead, the version that is currently released, follow all the steps in the above-mentioned link except for: “Install TensorFlow with Anaconda”, DO NOT follow guide from this point onward. instead :
After F1 -as we have now installed anaconda open anaconda and Create an environment: To do so open anaconda prompt from the start menu, its a terminal similar to cmd. MAKE SURE everything is in the path before proceeding, including CUDA. To check if Cuda is in path or not type: nvcc --version (in anaconda prompt) this will display the CUDA version. ANACONDA PROMPT:
conda create — — name wTF python=3.6Here wTF is the name of the environment we intend to create(workTensorFlow) and followed by a version of python we need i.e 3.6.
Press ‘y’ and sit back and feel something along the lines of :
It will install all relevant libraries to its knowledge and it won't be enough. The benefit of having/creating an ENVIRONMENT. As mentioned before there are many libraries that work in a cohort, anyone library is out of date and the whole tower comes down tumbling. In order to maintain Isolation from any other project we might be indulged in, we create an isolated environment that will contain the damage and keep all files independent of other projects. That means you can have TensorFlow 1.14, TF-1.15, TF-2.0, TF-2.1 all in one machine with all of their own assisting libraries.
2. Now that we have our environment we need to activate it to work inside it, this needs to be done every time anaconda prompt is used(or can be done from the drop-down menu if you are running IDE on that env). To see all the envs present in the conda type- conda info — -envs this will list all the envs created/present on the system. Type: conda activate wTF to activate the env and then install libraries within the env.
3. Pip- Pip is again a collection of packages sorted in a nice fashion which is easy to install/ uninstall using PIP, but 1st we need to install pip. to do that follow this tutorial. Pip makes life a lot easier.
4. Let's install TF-1.14 with the command:
conda install -c conda-forge tensorflow-gpu=1.14
Press y and wait yet again this time for a longer period of time depending on internet speed.
5. We have left some more additional libs which will help us to open and save CSV files among many other tasks. To install those libs I have provided link to a txt doc here, download the txt named req_pip.txt and copy it in any folder, open anaconda prompt and browse to the location of req_pip.txt then type:
pip install -r req_pip.txt
3rd time’s a charm, wait and let it all installed. type: pip list to see the list of all the libs installed.
Last checklist command is to ask TensorFlow whether it recognizes our GPU or not, to do that we will run python followed by importing tensor-flow and then asking for device.:( run the below command from within the python)
import tensorflow as tftf.test_is_gpu_available()
should return :
Created TensorFlow device (/device:GPU:0 with 4616 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2060, pci bus id: 0000:01:00.0, compute capability: 7.5)
TrueThis means Congratulations are in order, you have successfully completed the most tiresome part of the job. Anaconda Navigator will List the ENVS just select the one you created and open the SPYDER.
Article 2 will cover TENSOR-FLOW 1.14 model garden setup and running Inference on images and Videos.
