Photo Voltiac Viability Map I

This is a follow up to my previous post titled Getting Started with LIDAR Data written on the 6th of May.

I'll be building on the project that Sam Phillips completed during his 3 week internship.  My first task is to get Sam's application running on my machine.  I setup a VirtualBox VM for that purpose, using the same approach as I did to install django CMS on another VM.  I've already made my own fork of Sam's project.

Starting with a fresh VM running Ubuntu 14.04 server, I logged into it from my terminal using ssh

and did the following:

• $ sudo aptitude install git
• $ git clone git@github.com:jelkner/pv-viability-map.git

I got a:

Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

since I didn't have an ssh public key on github for the new VM. To fix that, I did:

• $ ssh-keygen
• $ cd .ssh
• Navigated to my profile on github and clicked the "Edit" button.
• Clicked on the "SSH Keys" link.
• Clicked the "Add SSH Key" button.
• Opened the id_rsa.pub file in the .ssh directory in a text editor.
• Copied and pasted the contents of this file to the "key" field on the github form (after giving it an appropriate title).

After doing this was able to rerun the git clone command and it downloaded the pv-viability-map source to my VM.

Sam did a terrific job documenting his work.  His README.md tells the user where to start, and his documention.txt file contains detailed information about how his application works.  That plus the source code is all I should need to get started.

Since I have neither make nor flask installed, I'll add those along with PostgreSQL:

• $ sudo aptitude install make python3-flask postgresql postgis

I'm going to need to spend some time learning PostgreSQL before I can continue too much further with this project, but before I stop to do that I'll try to start the server and see if I can find other missing dependencies.

• $ make run
  coffee -c sunlight.coffee
  make: coffee: Command not found
  make: *** [sunlight.js] Error 127

 OK, I need CoffeeScript.

• $ sudo aptitude install coffeescript
•  $ make run
  coffee -c sunlight.coffee
  python3 httpserver.py
  Traceback (most recent call last):
    File "httpserver.py", line 3, in <module>
      import api
    File "/home/[username]/pv-viability-map/api.py", line 1, in <module>
      import geojson
  ImportError: No module named 'geojson'
  make: *** [run] Error 1

geojson is a python module that doesn't have an ubuntu package available for it, so I'll need to use pip for that.  I can also see from the import section of api.py that I'm going to need shapely. There is an ubuntu package for that. I'll keep running make run and installing missing dependencies until I get a failure to connect to the database.  I'll also install a few of the libraries I needed for django CMS just for good measure.

• $ sudo aptitude install python3-pip python3-shapely
• $ sudo pip3 install geojson
• $ sudo aptitude install python3-all-dev libtiff5-dev libjpeg8-dev
• $ sudo aptitude install libwebp-dev tcl8.6-dev tk8.6-dev
• $ make run
  python3 httpserver.py
  Traceback (most recent call last):
    File "httpserver.py", line 4, in <module>
      import psycopg2
  ImportError: No module named 'psycopg2'
  make: *** [run] Error 1
• $ sudo aptitude install python3-psycopg2
• $ make run
  python3 httpserver.py
  Traceback (most recent call last):
    File "httpserver.py", line 6, in <module>
      import import_tool
    File "/home/jelkner/pv-viability-map/import_tool.py", line 8, in <module>
      import shapefile
  ImportError: No module named 'shapefile'
  make: *** [run] Error 1
• $ sudo aptitude install python3-pyshp
• $ make run
  python3 httpserver.py
  Traceback (most recent call last):
    File "httpserver.py", line 6, in <module>
      import import_tool
    File "/home/jelkner/pv-viability-map/import_tool.py", line 10, in <module>
      import pyproj
  ImportError: No module named 'pyproj'
  make: *** [run] Error 1
• $ sudo aptitude install python3-pyproj
• $ make run
  python3 httpserver.py
  Traceback (most recent call last):
    File "httpserver.py", line 6, in <module>
      import import_tool
    File "/home/jelkner/pv-viability-map/import_tool.py", line 11, in <module>
      from osgeo import osr
  ImportError: No module named 'osgeo'
  make: *** [run] Error 1
• $ sudo aptitude install python3-gdal
• $ make run
  python3 httpserver.py
  Traceback (most recent call last):
    File "httpserver.py", line 17, in <module>
      host=app.config["DBHOST"]
    File "/usr/lib/python3/dist-packages/psycopg2/__init__.py", line 179, in connect
      connection_factory=connection_factory, async=async)
  psycopg2.OperationalError: FATAL:  password authentication failed for user "sunlight"
  FATAL:  password authentication failed for user "sunlight"
  
  make: *** [run] Error 1

A rather tedious way to find all the dependencies, but I finally failed on a database error, so it's time to stop for now until I spend some time learning PostgreSQL.

I have to remember to clean this up later into a much more compact set of installation instructions. I'll also take the opportunity now to make a copy of my virtual hard drive, so I've got a snapshot of it in its present state.

---

Before ending this post, I want to note something about the data source we are using for the LIDAR data. The USGS has a website called EarthExplorer that  has the Loudoun County Virginia LIDAR data set in one large (76.7 megabytes) file. This is not ideal to use for experimentation and learning, since scripts which run on it will take a long time to run (we are using Loudoun County instead of Arlington County data because the Arlington County LIDAR data is not available yet).

A website called Virginia Lidar has the same data available in much smaller LAZ files, so I'll start with the data here.