AlphaPy: An AutoML Framework for Python

Discussion in 'App Development' started by PoundTheRock, Aug 19, 2017.

  1. Did you actually get this to build? Using Windows with VS2017 and Anaconda with Python 2.7, after building and installing XGBoost get all sorts of errors. Looks like it is looking for MKL, but perhaps I can replace that with OPENBLAS/LAPACK.

    Code:
    Building wheels for collected packages: scipy, statsmodels, scandir, bottleneck
      Running setup.py bdist_wheel for scipy ... error
      Complete output from command C:\ProgramData\Anaconda2\python.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\admini~1\\appdata\\local\\temp\\2\\pip-build-weu8re\
    cipy\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d c:\users\admini
    \appdata\local\temp\2\tmpfoikqjpip-wheel- --python-tag cp27:
      lapack_opt_info:
      lapack_mkl_info:
      libraries mkl_rt not found in ['C:\\ProgramData\\Anaconda2\\lib', 'C:\\', 'C:\\ProgramData\\Anaconda2\\libs']
      NOT AVAILABLE
    
      openblas_lapack_info:
      libraries openblas not found in ['C:\\ProgramData\\Anaconda2\\lib', 'C:\\', 'C:\\ProgramData\\Anaconda2\\libs']
      NOT AVAILABLE
    
      atlas_3_10_threads_info:
      Setting PTATLAS=ATLAS
      C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\system_info.py:1051: UserWarning: Specified path C:\projects\numpy-wheels\windows-wheel-builder\atlas-builds\atlas
    .11.38-sse2-64\lib is invalid.
      pre_dirs = system_info.get_paths(self, section, key)
      <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
      NOT AVAILABLE
    
      atlas_3_10_info:
      <class 'numpy.distutils.system_info.atlas_3_10_info'>
      NOT AVAILABLE
    
      atlas_threads_info:
      Setting PTATLAS=ATLAS
      <class 'numpy.distutils.system_info.atlas_threads_info'>
      NOT AVAILABLE
    
      atlas_info:
      <class 'numpy.distutils.system_info.atlas_info'>
      NOT AVAILABLE
    
      C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\system_info.py:572: UserWarning:
      Atlas (http://math-atlas.sourceforge.net/) libraries not found.
      Directories to search for the libraries can be specified in the
      numpy/distutils/site.cfg file (section [atlas]) or by setting
      the ATLAS environment variable.
      self.calc_info()
      lapack_info:
      libraries lapack not found in ['C:\\ProgramData\\Anaconda2\\lib', 'C:\\', 'C:\\ProgramData\\Anaconda2\\libs']
      NOT AVAILABLE
    
      C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\system_info.py:572: UserWarning:
      Lapack (http://www.netlib.org/lapack/) libraries not found.
      Directories to search for the libraries can be specified in the
      numpy/distutils/site.cfg file (section [lapack]) or by setting
      the LAPACK environment variable.
      self.calc_info()
      lapack_src_info:
      NOT AVAILABLE
    
      C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\system_info.py:572: UserWarning:
      Lapack (http://www.netlib.org/lapack/) sources not found.
      Directories to search for the sources can be specified in the
      numpy/distutils/site.cfg file (section [lapack_src]) or by setting
      the LAPACK_SRC environment variable.
      self.calc_info()
      NOT AVAILABLE
    
      Running from scipy source directory.
      non-existing path in 'scipy\\integrate': 'quadpack.h'
      Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "c:\users\admini~1\appdata\local\temp\2\pip-build-weu8re\scipy\setup.py", line 416, in <module>
      setup_package()
      File "c:\users\admini~1\appdata\local\temp\2\pip-build-weu8re\scipy\setup.py", line 412, in setup_package
      setup(**metadata)
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\core.py", line 135, in setup
      config = configuration()
      File "c:\users\admini~1\appdata\local\temp\2\pip-build-weu8re\scipy\setup.py", line 336, in configuration
      config.add_subpackage('scipy')
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\misc_util.py", line 1029, in add_subpackage
      caller_level = 2)
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\misc_util.py", line 998, in get_subpackage
      caller_level = caller_level + 1)
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\misc_util.py", line 935, in _get_configuration_from_setup_py
      config = setup_module.configuration(*args)
      File "scipy\setup.py", line 15, in configuration
      config.add_subpackage('linalg')
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\misc_util.py", line 1029, in add_subpackage
      caller_level = 2)
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\misc_util.py", line 998, in get_subpackage
      caller_level = caller_level + 1)
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\misc_util.py", line 935, in _get_configuration_from_setup_py
      config = setup_module.configuration(*args)
      File "scipy\linalg\setup.py", line 20, in configuration
      raise NotFoundError('no lapack/blas resources found')
      numpy.distutils.system_info.NotFoundError: no lapack/blas resources found
    
      ----------------------------------------
      Failed building wheel for scipy
      Running setup.py clean for scipy
      Complete output from command C:\ProgramData\Anaconda2\python.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\admini~1\\appdata\\local\\temp\\2\\pip-build-weu8re\
    cipy\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" clean --all:
    
      `setup.py clean` is not supported, use one of the following instead:
    
      - `git clean -xdf` (cleans all files)
      - `git clean -Xdf` (cleans all versioned files, doesn't touch
      files that aren't checked into the git repo)
    
      Add `--force` to your command to use it anyway if you must (unsupported).
    
    
      ----------------------------------------
      Failed cleaning build dir for scipy
      Running setup.py bdist_wheel for statsmodels ... done
      Stored in directory: C:\Users\Administrator\AppData\Local\pip\Cache\wheels\23\82\8a\d7f0a956c594584933bdedaacac68f7bc962020943496302a0
      Running setup.py bdist_wheel for scandir ... done
      Stored in directory: C:\Users\Administrator\AppData\Local\pip\Cache\wheels\47\af\a2\eb591a17e9709a17d8b53696f6ad89914a05eaf9c091f36e18
      Running setup.py bdist_wheel for bottleneck ... done
      Stored in directory: C:\Users\Administrator\AppData\Local\pip\Cache\wheels\4d\56\b2\36b69796ed42dbb94d921772e9d9dedeb90e62905b1069f09a
    Successfully built statsmodels scandir bottleneck
    Failed to build scipy
    Installing collected packages: pytz, pyparsing, matplotlib, scipy, seaborn, decorator, setuptools, prompt-toolkit, scandir, pathlib2, ipython-genutils, traitlets, colorama, py
    ents, ipython, statsmodels, pandas, scikit-learn, category-encoders, requests-file, pandas-datareader, bottleneck, empyrical, pyfolio, imbalanced-learn, alphapy
      Found existing installation: pytz 2016.10
      DEPRECATION: Uninstalling a distutils installed project (pytz) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a dis
    tils project will only partially uninstall the project.
      Uninstalling pytz-2016.10:
      Successfully uninstalled pytz-2016.10
      Found existing installation: pyparsing 2.1.4
      Uninstalling pyparsing-2.1.4:
      Successfully uninstalled pyparsing-2.1.4
      Found existing installation: matplotlib 2.0.0
      Uninstalling matplotlib-2.0.0:
      Successfully uninstalled matplotlib-2.0.0
      Found existing installation: scipy 0.18.1
      Uninstalling scipy-0.18.1:
      Successfully uninstalled scipy-0.18.1
      Running setup.py install for scipy ... error
      Complete output from command C:\ProgramData\Anaconda2\python.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\admini~1\\appdata\\local\\temp\\2\\pip-build-weu8r
    \scipy\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record c:\users\ad
    ni~1\appdata\local\temp\2\pip-hmqwt8-record\install-record.txt --single-version-externally-managed --compile:
    
      Note: if you need reliable uninstall behavior, then install
      with pip instead of using `setup.py install`:
    
      - `pip install .`  (from a git repo or downloaded source
      release)
      - `pip install scipy`  (last SciPy release on PyPI)
    
    
      lapack_opt_info:
      lapack_mkl_info:
      libraries mkl_rt not found in ['C:\\ProgramData\\Anaconda2\\lib', 'C:\\', 'C:\\ProgramData\\Anaconda2\\libs']
      NOT AVAILABLE
    
      openblas_lapack_info:
      libraries openblas not found in ['C:\\ProgramData\\Anaconda2\\lib', 'C:\\', 'C:\\ProgramData\\Anaconda2\\libs']
      NOT AVAILABLE
    
      atlas_3_10_threads_info:
      Setting PTATLAS=ATLAS
      C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\system_info.py:1051: UserWarning: Specified path C:\projects\numpy-wheels\windows-wheel-builder\atlas-builds\atl
    -3.11.38-sse2-64\lib is invalid.
      pre_dirs = system_info.get_paths(self, section, key)
      <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
      NOT AVAILABLE
    
      atlas_3_10_info:
      <class 'numpy.distutils.system_info.atlas_3_10_info'>
      NOT AVAILABLE
    
      atlas_threads_info:
      Setting PTATLAS=ATLAS
      <class 'numpy.distutils.system_info.atlas_threads_info'>
      NOT AVAILABLE
    
      atlas_info:
      <class 'numpy.distutils.system_info.atlas_info'>
      NOT AVAILABLE
    
      C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\system_info.py:572: UserWarning:
      Atlas (http://math-atlas.sourceforge.net/) libraries not found.
      Directories to search for the libraries can be specified in the
      numpy/distutils/site.cfg file (section [atlas]) or by setting
      the ATLAS environment variable.
      self.calc_info()
      lapack_info:
      libraries lapack not found in ['C:\\ProgramData\\Anaconda2\\lib', 'C:\\', 'C:\\ProgramData\\Anaconda2\\libs']
      NOT AVAILABLE
    
      C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\system_info.py:572: UserWarning:
      Lapack (http://www.netlib.org/lapack/) libraries not found.
      Directories to search for the libraries can be specified in the
      numpy/distutils/site.cfg file (section [lapack]) or by setting
      the LAPACK environment variable.
      self.calc_info()
      lapack_src_info:
      NOT AVAILABLE
    
      C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\system_info.py:572: UserWarning:
      Lapack (http://www.netlib.org/lapack/) sources not found.
      Directories to search for the sources can be specified in the
      numpy/distutils/site.cfg file (section [lapack_src]) or by setting
      the LAPACK_SRC environment variable.
      self.calc_info()
      NOT AVAILABLE
    
      Running from scipy source directory.
      non-existing path in 'scipy\\integrate': 'quadpack.h'
      Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "c:\users\admini~1\appdata\local\temp\2\pip-build-weu8re\scipy\setup.py", line 416, in <module>
      setup_package()
      File "c:\users\admini~1\appdata\local\temp\2\pip-build-weu8re\scipy\setup.py", line 412, in setup_package
      setup(**metadata)
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\core.py", line 135, in setup
      config = configuration()
      File "c:\users\admini~1\appdata\local\temp\2\pip-build-weu8re\scipy\setup.py", line 336, in configuration
      config.add_subpackage('scipy')
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\misc_util.py", line 1029, in add_subpackage
      caller_level = 2)
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\misc_util.py", line 998, in get_subpackage
      caller_level = caller_level + 1)
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\misc_util.py", line 935, in _get_configuration_from_setup_py
      config = setup_module.configuration(*args)
      File "scipy\setup.py", line 15, in configuration
      config.add_subpackage('linalg')
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\misc_util.py", line 1029, in add_subpackage
      caller_level = 2)
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\misc_util.py", line 998, in get_subpackage
      caller_level = caller_level + 1)
      File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\distutils\misc_util.py", line 935, in _get_configuration_from_setup_py
      config = setup_module.configuration(*args)
      File "scipy\linalg\setup.py", line 20, in configuration
      raise NotFoundError('no lapack/blas resources found')
      numpy.distutils.system_info.NotFoundError: no lapack/blas resources found
    
      ----------------------------------------
      Rolling back uninstall of scipy
    Command "C:\ProgramData\Anaconda2\python.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\admini~1\\appdata\\local\\temp\\2\\pip-build-weu8re\\scipy\\setup.py';f=ge
    ttr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record c:\users\admini~1\appdata\local\tem
    2\pip-hmqwt8-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in c:\users\admini~1\appdata\local\temp\2\pip-build-weu8re\scipy
    
    =======
    
    
     
    Last edited: Aug 19, 2017
  2. I got around it.

    • Use conda to install numpy and scipy, not pip. So install those "by hand" and their install system will allow the installed scipy and numpy.
    • You will also need setuptools which the system will not install for you.
    Let's see if this works.
     
    Last edited: Aug 19, 2017
  3. Ok not even their example works:

    Code:
      Directory: C:\Users\Administrator\Documents\python\AlphaPy\alphapy\examples\Trading Model
    
    
    Mode  LastWriteTime  Length Name
    ----  -------------  ------ ----
    d----  8/19/2017  8:01 PM  config
    -a---  8/19/2017  8:01 PM  13379 A Trading Model.ipynb
    
    PS C:\Users\Administrator\Documents\python\AlphaPy\alphapy\examples\Trading Model> mflow --pdate 2017-01-01
    Traceback (most recent call last):
      File "c:\programdata\anaconda2\lib\runpy.py", line 174, in _run_module_as_main
      "__main__", fname, loader, pkg_name)
      File "c:\programdata\anaconda2\lib\runpy.py", line 72, in _run_code
      exec code in run_globals
      File "C:\ProgramData\Anaconda2\Scripts\mflow.exe\__main__.py", line 5, in <module>
      File "c:\programdata\anaconda2\lib\site-packages\alphapy\market_flow.py", line 30, in <module>
      from alphapy.analysis import Analysis
      File "c:\programdata\anaconda2\lib\site-packages\alphapy\analysis.py", line 29, in <module>
      from alphapy.__main__ import main_pipeline
      File "c:\programdata\anaconda2\lib\site-packages\alphapy\__main__.py", line 32, in <module>
      from alphapy.estimators import get_estimators
      File "c:\programdata\anaconda2\lib\site-packages\alphapy\estimators.py", line 54, in <module>
      import xgboost as xgb
      File "c:\programdata\anaconda2\lib\site-packages\xgboost-0.6-py2.7.egg\xgboost\__init__.py", line 11, in <module>
      from .core import DMatrix, Booster
      File "c:\programdata\anaconda2\lib\site-packages\xgboost-0.6-py2.7.egg\xgboost\core.py", line 115, in <module>
      _LIB = _load_lib()
      File "c:\programdata\anaconda2\lib\site-packages\xgboost-0.6-py2.7.egg\xgboost\core.py", line 109, in _load_lib
      lib = ctypes.cdll.LoadLibrary(lib_path[0])
      File "c:\programdata\anaconda2\lib\ctypes\__init__.py", line 440, in LoadLibrary
      return self._dlltype(name)
      File "c:\programdata\anaconda2\lib\ctypes\__init__.py", line 362, in __init__
      self._handle = _dlopen(self._name, mode)
    WindowsError: [Error 193] %1 is not a valid Win32 application
    
     
  4. Whoops. For Anaconda, it looks like this requires Python 3.5! Missed that. Their Readme and their online docs are out of sync.

    Code:
    conda create -n alphapy python=3.5
    source activate alphapy
    conda install -c conda-forge bokeh
    conda install -c conda-forge ipython
    conda install -c conda-forge matplotlib
    conda install -c conda-forge numpy
    conda install -c conda-forge pandas
    conda install -c conda-forge pyyaml
    conda install -c conda-forge scikit-learn
    conda install -c conda-forge scipy
    conda install -c conda-forge seaborn
    conda install -c conda-forge xgboost
    pip install pandas_datareader
    pip install imbalanced-learn
    pip install category_encoders
    pip install pyfolio
    
     
  5. I followed the instructions right above this post. On XGBoost it comes back

    Code:
    PS C:\Users\Administrator\Documents\python> conda install -c conda-forge xgboost
    Fetching package metadata .............
    
    PackageNotFoundError: Packages missing in current channels:
    
      - xgboost
    
    We have searched for the packages in the following channels:
    
      - https://conda.anaconda.org/conda-forge/win-64
      - https://conda.anaconda.org/conda-forge/noarch
      - https://repo.continuum.io/pkgs/free/win-64
      - https://repo.continuum.io/pkgs/free/noarch
      - https://repo.continuum.io/pkgs/r/win-64
      - https://repo.continuum.io/pkgs/r/noarch
      - https://repo.continuum.io/pkgs/pro/win-64
      - https://repo.continuum.io/pkgs/pro/noarch
      - https://repo.continuum.io/pkgs/msys2/win-64
      - https://repo.continuum.io/pkgs/msys2/noarch
    
    I plowed forward anyway since I built XGBoost from source and installed it. Then tried the Trading Model example again, this time from the virtual python 3.5 environment. Same error.

    I give up for now.
     
  6. Hey PTR,
    Does Alphypy still work? I installed it but couldn't make it work. Examples folder is missing.
     
  7. Hi swinging tick (awesome nick btw :cool:):

    Thanks for your message.

    We took it private around the same time that Quantopian/Pyfolio went under, so the public version is not maintained any more... will probably archive at some point.

    Regards,

    PTR
     
  8. Alright, thanks for reply!
     
    PoundTheRock likes this.