|CyVerse_logo|_

Learning Center Home

Install Popular Data Science Tools on Atmosphere or Jetstream Instances

Goal

ez allows an “easy” installation of Anaconda based Jupyter notebook, Jupyter Hub, Singularity, and Docker.

This quickstart will cover specific commands for installing the following:

• Jupyter Notebooks
• Jupyter Hub
• Singularity
• Docker

---

|CyVerse_logo|_

Learning Center Home

RStudio-Server install with Anaconda

Setting up RStudio-Server with the `ezj -R` function on Atmosphere

Recently we set up a ez script which is executed in the terminal or web shell to install Anaconda3 with Jupyter Notebook. One of the features is to install R kernel with Jupyter. It also installs r-essentials with numerous common R packages.

1. Be sure to run ezj -R first. Reference documentation above if you have not. To set up RStudio-Server with the conda installation of R you need to set up the bash profile.

• Add anaconda3 to your path

export PATH="/opt/anaconda3/bin:$PATH"

• Change ownership of the /opt/anaconda3/ directory

sudo chown ${USER}:iplant-everyone /opt/anaconda3/ -R

• Add a path to your ~/.bash_profile:

echo "export RSTUDIO_WHICH_R='/opt/anaconda3/bin/R'" >> ~/.bash_profile

Note

You will need to exit and restart your terminal for these to take effect

2. Install RStudio-Server using the latest version

Note

This will fail on the first try:

user_name@128:/home$ sudo gdebi rstudio-server-1.0.143-amd64.deb
Reading package lists... Done
Building dependency tree
Reading state information... Done
Reading state information... Done

RStudio Server
RStudio is a set of integrated tools designed to help you be more productive with R. It includes a console, syntax    highlighting editor that supports direct code execution, as well as tools for plotting, history, and workspace management.
Do you want to install the software package? [y/N]:y
(Reading database ... 136874 files and directories currently installed.)
 Preparing to unpack rstudio-server-1.0.143-amd64.deb ...
 Unpacking rstudio-server (1.0.143) over (1.0.143) ...
 Setting up rstudio-server (1.0.143) ...
 useradd: user 'rstudio-server' already exists
 groupadd: group 'rstudio-server' already exists
 rsession: no process found
 Created symlink from /etc/systemd/system/multi-user.target.wants/rstudio-server.service to /etc/systemd/system/rstudio- server.service.
 Job for rstudio-server.service failed because the control process exited with error code. See "systemctl status rstudio- server.service" and "journalctl -xe" for details.
 ● rstudio-server.service - RStudio Server
 Loaded: loaded (/etc/systemd/system/rstudio-server.service; enabled; vendor preset: enabled)
 Active: active (running) since Sat 2017-05-13 09:30:40 MST; 13ms ago
   Process: 2226 ExecStop=/usr/bin/killall -TERM rserver (code=exited, status=1/FAILURE)
   Process: 2233 ExecStart=/usr/lib/rstudio-server/bin/rserver (code=exited, status=0/SUCCESS)
  Main PID: 2236 (rserver)
  Tasks: 3
  Memory: 824.0K
    CPU: 10ms
 CGroup: /system.slice/rstudio-server.service
         └─2236 /usr/lib/rstudio-server/bin/rserver

 May 13 09:30:40 xxx.xxx.xx.xxx systemd[1]: rstudio-server.service: Service hold-off time over, scheduling restart.
 May 13 09:30:40 xxx.xxx.xx.xxx systemd[1]: Stopped RStudio Server.
 May 13 09:30:40 xxx.xxx.xx.xxx systemd[1]: Starting RStudio Server...
 May 13 09:30:40 xxx.xxx.xx.xxx systemd[1]: Started RStudio Server.
 May 13 09:30:40 xxx.xxx.xx.xxx rserver[2236]: ERROR Unable to find an installation of R on the system (which R didn't return  va...pp:472
 May 13 09:30:40 xxx.xxx.xx.xxx systemd[1]: rstudio-server.service: Main process exited, code=exited, status=1/FAILURE
 Hint: Some lines were ellipsized, use -l to show in full.

3. Modify /etc/rstudio/rserver.conf

   sudo sh -c 'echo "rsession-which-r=/opt/anaconda3/bin/R" >> /etc/rstudio/rserver.conf'

4. Restart RStudio-Server

   sudo rstudio-server start

5. Log into RStudio-Server

   • Copy the IP address for the VM from the Atmosphere browser window.
   • Paste the IP address into a new browser window
   • add :8787 port # to the IP address
   • Log in using your CyVerse Username and Password.

Installing Packages for R and RStudio-Server

Note

Because we are using Anaconda3, it is suggested that you use conda to install your R packages from a terminal

Examples

conda install -c r r-raster

conda install -c conda-forge gdal

---

Summary

This documentation is intended for use with CyVerse Atmosphere featured images. It has been tested on Ubuntu 16.04 and Centos 6.8 images.

Additional information, help

---

Fix or improve this documentation

• Search for an answer: |CyVerse Learning Center|
• Ask us for help: click |Intercom| on the lower right-hand side of the page
• Report an issue or submit a change: |Github Repo Link|
• Send feedback: Tutorials@CyVerse.org

---

Learning Center Home

|CyVerse_logo|_

Learning Center Home

Advanced Installation of Docker

Docker-Compose Install

1. For instances without Anaconda install pip

sudo wget https://bootstrap.pypa.io/get-pip.py

sudo python3 get-pip.py

2. For instances with Anaconda (ezj) change ownership of /opt/anaconda3

On Atmosphere: sudo chown ${USER}:iplant-everyone /opt/anaconda3/ -R

On Jetstream: sudo chown ${USER}:root /opt/anaconda3/ -R

3. Install Docker-Compose

sudo pip3 install docker-compose

Running Docker without sudo

1. Check for and add the group docker (note: docker group should already exist)

groupadd docker

2. Add user to group

sudo usermod -aG docker $USER

3. Close Terminal and reopen
4. Test Docker without sudo

docker run hello-world

---

Fix or improve this documentation

• Search for an answer: |CyVerse Learning Center|
• Ask us for help: click |Intercom| on the lower right-hand side of the page
• Report an issue or submit a change: |Github Repo Link|
• Send feedback: Tutorials@CyVerse.org

---

Learning Center Home

---

Prerequisites

Downloads, access, and services

In order to complete this tutorial you will need access to the following services/software

Platform(s)

We will use the following CyVerse platform(s):

Platform	Interface	Link	Platform Documentation	Quick Start
Atmosphere	Command line (ssh) and/or Desktop (VNC, Guacamole WebDesktop)	Atmosphere	Atmosphere Manual	Guide
Jetstream	Command line (ssh) and/or Desktop (VNC, Guacamole WebDesktop)	Jetstream	Jetstream Manual	XSEDE Portal

Atmosphere Images

``ez`` is deployed on all featured instances in Atmosphere and Jetstream

---

Get Started

Launch an Atmosphere or Jetstream instance.

Connect via ssh using a terminal.

OR

Connect via the instance’s web shell via the Atmosphere Instance’s web page.

EZ Install Anaconda (Jupyter, Jupyter Lab, Jupyter Hub)

From your terminal session, you can install Anaconda Jupyter using the following

commands:

• Jupyter notebooks with Python 3 (default)

  ezj

• Jupyter notebooks with Python 2

  ezj -2

• Jupyter notebooks with R Kernel

  ezj -R or ezj -r

• Jupyter Hub

  ezjh

Jupyter Notebooks

After the ezj installation, you will be provided a URL (e.g. http ://128.196.65.162:8888/?token=2d6c40a7c8ee4b4933eaae5898101846bbfcd1e5d6bae37b) in your terminal session. Copy paste this into your new browser tab.

Note

A Jupyter Notebook is running as an active process in the foreground on your Atmosphere instance. If you disconnect from your Atmosphere terminal session, the Jupyter Notebook will terminate.

One trick to keep the session running is to use a screen or tmux and disconnect the session before you close your browser tab.

To terminate your Jupyter Notebook, close the browser page with the Jupyter notebook interface. In your Atmosphere ssh session, press: control + C to terminate the Jupyter notebook.

Tip

Restart a new Jupyter session on a VM with EZ already installed

At the terminal command prompt retype ezj, this will restart the conda virtual environment and a new Jupyter notebook. Connect to the notebook using the URL as in the instructions above. Don’t forget to use tmux!

Jupyter Lab (new)

Complete the ezj installation.

Jupyter Lab (Beta) is now included in the base Anaconda installation

To start a new Jupyter Lab session type:

jupyter lab

Install Jupyter Hub (new)

From terminal type ezjh

The installation may take up to 10 minutes to complete.

Once the install is complete, and a session is running, you’ll get a URL address to the VM.

Copy paste the URL into your new browser tab.

You will be re-directed back to a CyVerse CAS service, log into your account.

The Jupyter Hub should now be loaded in the browser tab.

EZ Install Singularity

From your terminal, type the following command: ezs

You should see

Updating ez singularity and installing singularity (this may take a few minutes, coffee break!)
[sudo] password for YourCyVerseUserName:

Wait for the installation to complete.

Test Singularity

singularity run shub://vsoch/hello-world

EZ Install Docker

1. `Connect to Atmosphere via ssh`_.

2. From your connected session, you can install Docker using the following command:

   ezd

3. Test Docker

   sudo docker run hello-world

Note

You need to use sudo permissions with Docker. After using the sudo command, Atmosphere will ask you for your CyVerse password for security purposes. You can remove the need to use sudo with Docker commands in the Advanced Docker Setup section.

---

Next Steps:

Some common next steps include:

1. Installation of RStudio on an Atmosphere Instance
2. Advanced Docker Setup
3. Adding to Anaconda

---

Additional Information

---

Fix or improve this documentation

• Search for an answer: |CyVerse Learning Center|
• Ask us for help: click |Intercom| on the lower right-hand side of the page
• Report an issue or submit a change: |Github Repo Link|
• Send feedback: Tutorials@CyVerse.org

---

Learning Center Home