【1.6】JupyterHub——让Jupyter支持多用户

June 06, 2018 py_basic 阅读量：次

支持多用户的jupyter

一、安装

1.下载并安装Anaconda3

wget -c https://repo.anaconda.com/archive/Anaconda3-5.2.0-Linux-x86_64.sh
bash Anaconda3-5.2.0-Linux-x86_64.sh

安装提示：

Do you wish the installer to prepend the Anaconda3 install location
to PATH in your /root/.bashrc ? [yes|no]
[no] >>> /data/user/sam/.bashrc

修改环境变量

vim /etc/profile

# Anaconda 3
#export PATH=/data/software/anaconda3/bin:$PATH  #因为现在默认的是anaconda2里面的python2
alias activate3="source /data/software/anaconda3/bin/activate root"
alias deactivate3="source /data/software/anaconda3/bin/deactivate"

source /etc/profile

2.下载安装 jupyterhub

source /data/software/anaconda3/bin/activate root  #启动conda环境,如无提示，后续的操作，都需要在该环境下完成

conda install -c conda-forge jupyterhub

3、创建配置文件

# source /data/software/anaconda3/bin/activate root 在conda环境下运行
jupyterhub --generate-config

提示：此时的工作路径是/data/software
Writing default config to: jupyterhub_config.py

4、修改配置文件

vim jupyterhub_config.py

取消注释，根据注释提示，填写相关信息

c.JupyterHub.ip = 'IP地址'
c.Spawner.ip = '127.0.0.1'
c.JupyterHub.port = 6364
c.PAMAuthenticator.encoding = 'utf8'
c.LocalAuthenticator.create_system_users = True
c.Authenticator.whitelist = {'user1', 'user1', 'user3'}
c.Authenticator.admin_users = {'user1'}
c.LocalAuthenticator.group_whitelist = {'group1'}
c.JupyterHub.statsd_prefix = 'jupyterhub'

组白名单好像会让用户白名单失效。所以需要确保用户在组白名单里面，需要检查一下/etc/group

5、启动JupyterHub

# source /data/software/anaconda3/bin/activate root 在conda环境下运行
nohup jupyterhub --no-ssl > jupyterhub.log &

（在配置文件目录下执行此命令）

8、测试访问

用IP+端口测试访问

9、用户管理

用户白名单的用户会自动添加，但无密码，需要修改密码才能登录；

adduser等

二、讨论

2.1.同时支持Python2和python3

Notebook的右上角点new 只看到 python 3 kernel,需要同时支持Python2和python3

查看目前的conda环境中的kernels

[root@pydev pydev]# jupyter-kernelspec list 
Available kernels: 
  python3    /data/software/anaconda3/share/jupyter/kernels/python3

查看是否安装了python2

python2 -m pip --version

然后，在该conda环境下通过Python2关联ipykernel

python2 -m pip install --upgrade ipykernel 
python2 -m pip install ipykernel
python2 -m ipykernel install --user

source activate anaconda2   ##这个是启动py2的conda环境
ipython kernel install

查看该环境下的kernels

# source /data/software/anaconda3/bin/activate root 在conda环境下运行
jupyter-kernelspec list
Available kernels:
  python2    /root/.local/share/jupyter/kernels/python2
  python3    /data/software/anaconda3/share/jupyter/kernels/python3

重启，打开jupyterhub，就能看到py2和py3的切换了

2.2 同步Linux的环境变量

jupyterhub启动以后，linux的path中新增的部分，发现没法在os.environ[‘PATH’]找到，最后不得已重启jupyterhub。

ps -ef |grep jupy
kill -9 process_id11111

cd /data/sotware # 在配置文件目录下执行
source /data/software/anaconda3/bin/activate root 

nohup jupyterhub --no-ssl > jupyterhub.log &

同时，修改配置文件，限制单个用户的环境变量不要出现

vim /data/software/jupyterhub_config.py

c.Spawner.env_keep = ['PATH', 'PYTHONPATH', 'CONDA_ROOT', 'CONDA_DEFAULT_ENV', 'VIRTUAL_ENV', 'LANG', 'LC_ALL']

##这里是想要单个用户中出现的环境变量
c.Spawner.environment = {}

三、报错

报错1 ： configurable-http-proxy没有访问权限

解决办法：

升级: npm install -g configurable-http-proxy

(base) [sam@c01 software]# jupyterhub --no-ssl
[I 2018-12-10 16:35:39.200 JupyterHub app:1667] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.2
[I 2018-12-10 16:35:39.200 JupyterHub app:1667] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.2
[I 2018-12-10 16:35:39.210 JupyterHub app:1014] Loading cookie_secret from /mnt/nfs/data/software/jupyterhub_cookie_secret
[I 2018-12-10 16:35:39.355 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN
[W 2018-12-10 16:35:39.361 JupyterHub auth:490] Ignoring username whitelist because group whitelist supplied!
[I 2018-12-10 16:35:39.583 JupyterHub app:1849] Hub API listening on http://127.0.0.1:8081/hub/
[W 2018-12-10 16:35:39.588 JupyterHub proxy:560] Running JupyterHub without SSL.  I hope there is SSL termination happening somewhere else...
[I 2018-12-10 16:35:39.588 JupyterHub proxy:562] Starting proxy @ http://*******25:6364/
[C 2018-12-10 16:35:39.601 JupyterHub app:1861] Failed to start proxy
    Traceback (most recent call last):
      File "/data/software/anaconda3/lib/python3.6/site-packages/jupyterhub/app.py", line 1859, in start
        await self.proxy.start()
      File "/data/software/anaconda3/lib/python3.6/site-packages/jupyterhub/proxy.py", line 566, in start
        self.proxy_process = Popen(cmd, env=env, start_new_session=True, shell=shell)
      File "/data/software/anaconda3/lib/python3.6/subprocess.py", line 709, in __init__
        restore_signals, start_new_session)
      File "/data/software/anaconda3/lib/python3.6/subprocess.py", line 1344, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    PermissionError: [Errno 13] Permission denied: 'configurable-http-proxy'

(base) [root@c01 software]# npm install -g configurable-http-proxy

/mnt/nfs/data/software/anaconda3/bin/configurable-http-proxy -> /mnt/nfs/data/software/anaconda3/lib/node_modules/configurable-http-proxy/bin/configurable-http-proxy
+ configurable-http-proxy@4.0.1
added 33 packages and updated 7 packages in 11.264s

   ╭─────────────────────────────────────╮
   │                                     │
   │   Update available 5.6.0 → 6.4.1    │
   │       Run npm i npm to update       │
   │                                     │

报错2：（未解决）

(base) [sam@c01 software]$ jupyterhub --no-ssl
[I 2019-03-04 11:33:20.137 JupyterHub app:1667] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.2
[I 2019-03-04 11:33:20.137 JupyterHub app:1667] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.2
[I 2019-03-04 11:33:20.400 JupyterHub app:1014] Loading cookie_secret from /mnt/nfs/data/software/jupyterhub_cookie_secret
[I 2019-03-04 11:33:22.226 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN
[W 2019-03-04 11:33:22.286 JupyterHub auth:490] Ignoring username whitelist because group whitelist supplied!
[I 2019-03-04 11:33:23.390 JupyterHub app:1849] Hub API listening on http://127.0.0.1:8081/hub/
[W 2019-03-04 11:33:23.403 JupyterHub proxy:472] Found proxy pid file: /mnt/nfs/data/software/jupyterhub-proxy.pid
[W 2019-03-04 11:33:23.470 JupyterHub proxy:489] Proxy still running at pid=56607
[W 2019-03-04 11:33:24.473 JupyterHub proxy:504] Stopped proxy at pid=56607
[W 2019-03-04 11:33:24.502 JupyterHub proxy:560] Running JupyterHub without SSL.  I hope there is SSL termination happening somewhere else...
[I 2019-03-04 11:33:24.502 JupyterHub proxy:562] Starting proxy @ http://19***:6364/
[C 2019-03-04 11:33:35.330 JupyterHub app:1861] Failed to start proxy
    Traceback (most recent call last):
      File "/data/software/anaconda3/lib/python3.6/site-packages/jupyterhub/app.py", line 1859, in start
        await self.proxy.start()
      File "/data/software/anaconda3/lib/python3.6/site-packages/jupyterhub/proxy.py", line 596, in start
        await server.wait_up(1)
      File "/data/software/anaconda3/lib/python3.6/site-packages/jupyterhub/utils.py", line 165, in wait_for_server
        timeout=timeout
      File "/data/software/anaconda3/lib/python3.6/site-packages/jupyterhub/utils.py", line 155, in exponential_backoff
        raise TimeoutError(fail_message)
    TimeoutError: Server at 192.168.50.125:6364 didn't respond in 1 seconds


(base) [sam@c01 software]$ 11:33:37.915 [ConfigProxy] info: Proxying http://1*****:6364 to (no default)
11:33:37.925 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes

解决路径：

修改了start_timeout

vim /data/software/jupyterhub_config.py
c.Spawner.start_timeout = 60

接着报错；

14:20:47.424 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes
14:20:48.738 [ConfigProxy] error: 404 GET /
14:20:48.760 [ConfigProxy] error: Failed to get custom error page: Error: connect ECONNREFUSED 127.0.0.1:8081
14:20:48.798 [ConfigProxy] error: 404 GET /favicon.ico
14:20:48.801 [ConfigProxy] error: Failed to get custom error page: Error: connect ECONNREFUSED 127.0.0.1:8081

关闭已开启的configurable-http-proxy服务

(base) [sam@c01 software]$ ps -ef |grep configurable-http-proxy
sam      30428     1  0 14:20 ?        00:00:01 node /data/software/anaconda3/bin/configurable-http-proxy --ip 1****5 --port 6364 --api-ip 127.0.0.1 --api-port 8001 --error-target http://127.0.0.1:8081/hub/error

[root@c01 ~]# netstat -lnp |grep 8001
tcp        0      0 127.0.0.1:8001          0.0.0.0:*               LISTEN      25354/node

kill -9 30428

可是还是一直报错，无力解决了，最后切换了一台服务器，就神奇般的好了，真是amazing !

报错3

[D 2017-08-29 17:27:43.001 JupyterHub proxy:490] Proxy started and appears to be up
[D 2017-08-29 17:27:43.003 JupyterHub proxy:551] Proxy: Fetching GET http://127.0.0.1:8001/api/routes
17:27:43.041 - warn: [ConfigProxy] 403 GET /api/routes 
[E 2017-08-29 17:27:43.040 JupyterHub app:1626]
    Traceback (most recent call last):
      File "/home/fmilano/jupyterhub/jupyterhub/app.py", line 1624, in launch_instance_async
        yield self.start()
      File "/home/fmilano/jupyterhub/jupyterhub/app.py", line 1572, in start
        yield self.proxy.check_routes(self.users, self._service_map)
      File "/home/fmilano/jupyterhub/jupyterhub/proxy.py", line 293, in check_routes
        routes = yield self.get_all_routes()
      File "/home/fmilano/jupyterhub/jupyterhub/proxy.py", line 588, in get_all_routes
        resp = yield self.api_request('', client=client)
    tornado.httpclient.HTTPError: HTTP 403: Forbidden

解决办法：

ps aux | grep configurable-http-proxy

然后，Kill掉以后，重启服务

报错4

登陆，提示用户名不存在或者密码不对。

检查密码是否正确，重置密码以后发现仍然不能登陆
检查/data/software/jupyterhub.log 文件，提示用户不在白名单中
/data/software/jupyterhub_config.py中的whitelist中加入该用户，发现依旧提示用户不在白名单中。
确保用户名所在的用户组为group_whitelist中的用户，确保/etc/group中的用户组中，包含该用户。问题解决。
上述如果都没问题，建议用root权限来jupyterhub

[#root] source /data/software/anaconda3/bin/activate root 在conda环境下运行 [#root] nohup jupyterhub –no-ssl > jupyterhub.log &

这个问题主要原因是，用户不在/etc/group中对应的用户组里面，而jupyterhub中的白名单组会取代白名单用户，只有白名单组生效。。

报错5

不提示错误，但一直不能访问。

有可能是防火墙打开，把端口给禁掉了。

报错6

500 : Internal Server Error

解决办法：

输入如下命令，安装升级nbconvert

pip install --upgrade --user nbconvert

成功安装完nbconvert后再次启动jupyter notebook,在浏览器就可以正常打开.ipynb文件。

发现还是有报错：

PermissionError: [Errno 13] Permission denied: '/data/software/miniconda3/share/jupyter/nbconvert/templates/html/conf.json'

发现是这个文件夹权限的问题：

[root@c01 ~]# chmod 777 /data/software/miniconda3/share/jupyter/nbconvert/templates/html

大公告及

参考资料

药企，独角兽，苏州。团队长期招人，感兴趣的都可以发邮件聊聊：tiehan@sina.cn

个人公众号，比较懒，很少更新，可以在上面提问题，如果回复不及时，可发邮件给我： tiehan@sina.cn

一、安装

1.下载并安装Anaconda3

2.下载安装 jupyterhub

3、创建配置文件

4、修改配置文件

5、启动JupyterHub

8、测试访问

9、用户管理

二、讨论

2.1.同时支持Python2和python3

2.2 同步Linux的环境变量

三、报错

报错1 ： configurable-http-proxy没有访问权限

报错2： （未解决）

报错3

报错4

报错5

报错6

参考资料

报错2：（未解决）