Creating executables for machine learning Python scripts
In my previous article, I have showed you how to create an .exe file using PyInstaller. However, it will be a bit of a challenge if your project uses too many libraries (which took me two days to solve the issues I encountered, hence thought of providing a step by step guide for Python users to save their valuable time) . Therefore, in this article I am focusing on creating .exe files for Python scripts for machine learning projects, where libraries such as pandas, numpy, scikit-learn are being used.
Here you go..
In my application, I am using following Python libraries;
1. Pandas
2. Numpy
3. Scikit-learn
4. Scipy
Most of the times, PyInstaller doesn't recognise all of these libraries. In this case you need to specify these libraries in hooks files one by one. How to do this? let's start from Pandas.
Note: Please install PyInstaller (follow the instructions given here), and keep your Python scripts ready.
1. Pandas
You can locate all the hooks files in the hooks folder of the PyInstaller distribution folder and see the names of the packages for which hooks have been written. In my case, all the hooks files are located in: ....\AppData\Local\Programs\Python\Python36-32\Lib\site-packages\PyInstaller\hooks.
Check whether it has the file named, hook-pandas.py, if not, create a one and add the following line.
hiddenimports = ['pandas._libs.tslibs.timedeltas'].
Now when you run the PyInstaller you will not get any error for Pandas.
2. Scikit-learn
After solving the module not found error for Pandas, I have got errors in different library files of Scikit-learn.
Therefore, after going through online help, and after countless trial and error efforts, creating following hooks files solved Scikit-learn issues.
(1). Create hook-sklearn.tree.py and add the following
hiddenimports = ['sklearn.tree._utils']
(2). Create hook-sklearn.metrics.cluster.py (if not exists) and add the following
hiddenimports = ['sklearn.utils.sparsetools._graph_validation',
'sklearn.utils.sparsetools._graph_tools',
'sklearn.utils.lgamma',
'sklearn.utils.weight_vector',
'sklearn.utils.fixes',
'sklearn.utils.extmath',
'sklearn.metrics.ranking',
'sklearn.neighbors',
'sklearn.neighbors.typedefs',
'sklearn.neighbors.quad_tree']
If this file already exists, you can modify it by adding the missing libraries one by one according to the error you get when running PyInstaller.
In my case, I had to add 'sklearn.neighbors', 'sklearn.neighbors.typedefs', and 'sklearn.neighbors.quad_tree'.
3. Scipy
Create hooks-scipy.py and add the following lines. This will tell PyInstaller to load all the dlls.
import os
import scipy
import platform
binaries = []
if "windows" in platform.platform().lower():
dll_path = os.path.join(os.path.dirname(scipy.__file__), 'extra-dll', "*.dll")
binaries = [(dll_path, "dlls")]
hiddenimports = ['scipy._lib.%s' % m for m in ['messagestream', "_ccallback_c", "_fpumode"]]
By following the above steps to include each library using hooks files, solved issues I encountered when creating the .exe using PyInstaller. Now when I run the following command,
pyinstaller my_project_name.py, an .exe file is created in the dist folder. (Here, my_project_name.py is the main Python script which uses the above mentioned libraries).
The article here provides more details about hooks in Pyinstaller, this will be useful in case you face a different issue with Python libraries.
Here you go..
In my application, I am using following Python libraries;
1. Pandas
2. Numpy
3. Scikit-learn
4. Scipy
Most of the times, PyInstaller doesn't recognise all of these libraries. In this case you need to specify these libraries in hooks files one by one. How to do this? let's start from Pandas.
Note: Please install PyInstaller (follow the instructions given here), and keep your Python scripts ready.
1. Pandas
You can locate all the hooks files in the hooks folder of the PyInstaller distribution folder and see the names of the packages for which hooks have been written. In my case, all the hooks files are located in: ....\AppData\Local\Programs\Python\Python36-32\Lib\site-packages\PyInstaller\hooks.
Check whether it has the file named, hook-pandas.py, if not, create a one and add the following line.
hiddenimports = ['pandas._libs.tslibs.timedeltas'].
Now when you run the PyInstaller you will not get any error for Pandas.
2. Scikit-learn
After solving the module not found error for Pandas, I have got errors in different library files of Scikit-learn.
Therefore, after going through online help, and after countless trial and error efforts, creating following hooks files solved Scikit-learn issues.
(1). Create hook-sklearn.tree.py and add the following
hiddenimports = ['sklearn.tree._utils']
(2). Create hook-sklearn.metrics.cluster.py (if not exists) and add the following
hiddenimports = ['sklearn.utils.sparsetools._graph_validation',
'sklearn.utils.sparsetools._graph_tools',
'sklearn.utils.lgamma',
'sklearn.utils.weight_vector',
'sklearn.utils.fixes',
'sklearn.utils.extmath',
'sklearn.metrics.ranking',
'sklearn.neighbors',
'sklearn.neighbors.typedefs',
'sklearn.neighbors.quad_tree']
If this file already exists, you can modify it by adding the missing libraries one by one according to the error you get when running PyInstaller.
In my case, I had to add 'sklearn.neighbors', 'sklearn.neighbors.typedefs', and 'sklearn.neighbors.quad_tree'.
3. Scipy
Create hooks-scipy.py and add the following lines. This will tell PyInstaller to load all the dlls.
import os
import scipy
import platform
binaries = []
if "windows" in platform.platform().lower():
dll_path = os.path.join(os.path.dirname(scipy.__file__), 'extra-dll', "*.dll")
binaries = [(dll_path, "dlls")]
hiddenimports = ['scipy._lib.%s' % m for m in ['messagestream', "_ccallback_c", "_fpumode"]]
By following the above steps to include each library using hooks files, solved issues I encountered when creating the .exe using PyInstaller. Now when I run the following command,
pyinstaller my_project_name.py, an .exe file is created in the dist folder. (Here, my_project_name.py is the main Python script which uses the above mentioned libraries).
The article here provides more details about hooks in Pyinstaller, this will be useful in case you face a different issue with Python libraries.
I gathered lots of information from your blog and it helped me a lot. Keep posting more.
ReplyDeletePython Training in Chennai
ccna course in Chennai
R Programming Training in Chennai
AWS Training in Chennai
DevOps Training in Chennai
Angularjs Training in Chennai
RPA Training in Chennai
Data Science Course in Chennai
This comment has been removed by the author.
ReplyDeleteVery impressive blog! I like much it and it was very helpful for me. Do share more ideas regularly.
ReplyDeleteCyber Security Training Course in Chennai | Certification | Cyber Security Online Training Course | Ethical Hacking Training Course in Chennai | Certification | Ethical Hacking Online Training Course |
CCNA Training Course in Chennai | Certification | CCNA Online Training Course | RPA Robotic Process Automation Training Course in Chennai | Certification | RPA Training Course Chennai | SEO Training in Chennai | Certification | SEO Online Training Course
After I at first remarked I appear to have tapped on the - Notify me when new remarks are added-checkbox and now every time a remark is added I get 4 messages with precisely the same remark. There must be a simple strategy you can eliminate me from that help? Credit!
ReplyDeletetech news
Hi! I simply need to offer you a major approval for your incredible data you have here on this post. I'll be returning to your site for all the more soon.
ReplyDeletebest interiors
This post is so interactive and informative.keep updating more information...
ReplyDeleteScope Of Java Developer
Career In Java
mmorpg
ReplyDeleteINSTAGRAM TAKİPCİ SATİN AL
tiktok jeton hilesi
tiktok jeton hilesi
antalya saç ekimi
referans kimliği nedir
İnstagram takipçi satın al
Mt2 pvp serverlar
İnstagram Takipci Satın Al
ataşehir vestel klima servisi
ReplyDeletetuzla samsung klima servisi
ataşehir bosch klima servisi
çekmeköy arçelik klima servisi
kadıköy samsung klima servisi
kadıköy mitsubishi klima servisi
tuzla lg klima servisi
tuzla alarko carrier klima servisi
tuzla daikin klima servisi
شركة شراء اثاث مستعمل
ReplyDeleteشركة تسليك مجاري
Superb blog post.
ReplyDeletePython classes in Pune