Category: visualization

The goal is to set up a Django management command and API route (with authentication support) for deleting HiGlass tilesets.

We will do the work on a t2.micro EC2 instance running Ubuntu 18.04 (ami-0f65671a86f061fcd).

Nodejs

Installation

Install node, npm, and npx from Nodejs:

$ cd ~
$ wget -qO- https://nodejs.org/dist/v11.1.0/node-v11.1.0-linux-x64.tar.xz > node-v11.1.0-linux-x64.tar.xz
$ tar xvf node-v11.1.0-linux-x64.tar.xz
...
$ cd node-v11.1.0-linux-x64/bin
$ sudo ln -s ${PWD}/node /usr/bin/node
$ sudo ln -s ${PWD}/npm /usr/bin/npm
$ sudo ln -s ${PWD}/npx /usr/bin/npx

Anaconda

Installation

$ cd ~
$ wget https://repo.anaconda.com/archive/Anaconda3-5.3.0-Linux-x86_64.sh
$ chmod +x Anaconda3-5.3.0-Linux-x86_64.sh
$ ./Anaconda3-5.3.0-Linux-x86_64.sh
...

Environment

Python 3.7 does not appear to be compatible with Cython at this time, so we downgrade to 3.6:

$ conda create -n higlass-server python=3.6 --no-default-packages --yes
...

Cf. https://github.com/cython/cython/issues/1978

Github

Clone and branch

We clone a fork of higlass-server, sync up with the upstream repository, and set up our branch off of the freshly-updated develop branch:

$ git clone https://github.com/alexpreynolds/higlass-server
$ cd ~/higlass-server
$ git remote add upstream https://github.com/higlass/higlass-server.git
$ git checkout develop
$ git pull remote develop
$ git checkout -b delete-tileset develop

Clean up old hms-dbmi references:

$ cd ~/higlass-server
$ grep -rl hms-dbmi . | xargs sed -i 's/hms-dbmi/higlass/g'

Development

Environment

Install current GCC kit and libraries:

$ sudo apt install build-essential
$ sudo apt install libglib2.0-dev
$ sudo apt install libbz2-dev
$ sudo apt install liblzma-dev
$ sudo apt install libhdf5-serial-dev
$ sudo apt install libcurl4-gnutls-dev
$ sudo apt install libpng-dev
$ sudo apt install libssl-dev
$ gcc --version
...

Python requirements

Note: Edit requirements-secondary.txt to build clodius v0.9.3, or newer.

$ cd ~/higlass-server
$ source activate higlass-server
(higlass-server) $ pip install --upgrade -r ./requirements.txt
(higlass-server) $ pip install --upgrade -r ./requirements-secondary.txt

Initialize server

(higlass-server) $ python manage.py makemigrations
(higlass-server) $ python manage.py migrate
(higlass-server) $ python manage.py runserver localhost:8000
...

Test API

From another terminal session:

$ wget -qO- http://localhost:8000/api/v1/tilesets
{"count":0,"next":null,"previous":null,"results":[]}
$ wget -qO- http://localhost:8000/api/v1/tileset_info/?d=1234
{"1234": {"error": "No such tileset with uid: 1234"}}

Test ingestion and listing

$ wget -O- --user * --password * https://resources.altius.org/~areynolds/LN43287.75_20.normalized.GRCh38_no_alts.bw.tileset > /tmp/LN43287.75_20.normalized.GRCh38_no_alts.bw.tileset
$ python manage.py ingest_tileset --filetype hitile --datatype vector --filename /tmp/LN43287.75_20.normalized.GRCh38_no_alts.bw.tileset
uid: AIVpsJYwSemD8FVPBv6vrw
$ python manage.py list_tilesets
tileset: Tileset [name: LN43287.75_20.normalized.GRCh38_no_alts.bw.tileset] [ft: hitile] [uuid: AIVpsJYwSemD8FVPBv6vrw]

Set up superuser

$ python manage.py createsuperuser
...

Pull request

https://github.com/higlass/higlass-server/pull/79

Read More

This assumes Homebrew is installed and that it installs Graphviz 2.40.1:

$ brew install graphviz
$ git clone https://github.com/pygraphviz/pygraphviz.git
$ cd pygraphviz
$ sudo python setup.py install --user --include-path=/usr/local/Cellar/graphviz/2.40.1/include --library-path=/usr/local/Cellar/graphviz/2.40.1/lib

Read More

Cartograms with d3   TopoJSON   Same Sex Marriage

Shawn Allen wrote a d3.js-based implementation of a 2D cartogram, which sizes US states in an area-proportional manner, where area is based on some interesting statistic, like population.

There has been a great deal of progress made in the last year in defending the rights of GLBT Americans to marry and have their partnership rights acknowledged, rights like visitation and estate planning, rights that straight couples take for granted when visiting their loved one in the hospital, or sharing their lives in the house they own, etc.

It’s easy enough to see a map of the 50 states colored by legal status, but people are not spread out evenly to live across all states. I wanted to see how the United States was progressing as a factor of population.

I forked Allen’s project (GitHub project source code available here) and I redid the color scheme, which takes the 50 states and the District of Columbia and shades them by their legal status, whether their laws defend or remove same-sex marriage rights (and associated protections).

Green states allow same-sex marriage, light-green states allow civil unions, orange allow marriage or civil unions (but rulings are currently held up on appeal), and red states that do not defend same-sex marriage rights, either by explicit law or constitutional amendment.

I based the color assignments initially on data from the Right to Marry site, up-to-date as of May 19th, 2014. But with Pennsylvania’s Gov. Corbett conceding defeat and vowing not to appeal the ruling, I added Pennsylvania to the list of pro-equality states.

In addition to seeing how fast things have changed, what is also interesting is that drawing by area quickly shows that over half the country — by 2010 US Census population counts, at least — now enjoys (or will soon enjoy, pending appeals) legal protections that were once denied to a minority of Americans.

Read More

For scientific work, I have used matrix2png to make a nice PNG image from a text-formatted matrix of data values. PNG looks great on the web, but it doesn’t translate well to making publication-quality figures.

My thought was to take matrix2png and — with the help of Haru (libharu) — turn it into matrix2pdf. Maybe I can get this going on Github.

Read More

I had recently updated my copy of R to 2.15.1 and ended up needing to reinstall some libraries, including rgl.

If you use this R library, it can be tricky to install with the built-in build of Mesa/OpenGL in Lion. In fact, a straightforward install.packages("rgl") just won’t work at all.

But after using MacPorts to install freeglut, I found the following R command helps build rgl successfully:

install.packages(“rgl”, repos=”http://R-Forge.R-project.org”, configure.args=”–disable-cocoa –with-gl-includes=/opt/local/include –with-gl-libs=/opt/local/lib –with-x”)

Once installed, rgl can make cool figures, like this:

Read More