[TOC]
问题原因
突然不能克隆base环境了
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
rm -rf pyspark3602; conda create --name pyspark3602 --clone base
Source: /home/hdp_lbg_ectech/wangke/app/anaconda3
Destination: /home/hdp_lbg_ectech/wangke/app/anaconda3/envs/pyspark3602
The following packages cannot be cloned out of the root environment:
- http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64::conda-4.9.2-py36h5fab9bb_0
- defaults/linux-64::conda-build-3.10.5-py36_0
Packages: 195
Files: 64657
...
CondaHTTPError: HTTP 404 NOT FOUND for url <http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/keras-applications-1.0.8-py_1.tar.bz2>
Elapsed: 00:00.351600
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
|
第一个问题
这里的主要问题是使用了清华的镜像. 清华镜像只有文件.conda后缀, 没有.tar.bz2后缀的文件,所以报CondaHTTPError: HTTP 404 NOT FOUND for url错误导致不能创建环境
http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/keras-applications-1.0.8-py_1.tar.bz2 不存在
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/keras-applications-1.0.8-py_1.conda 存在
第二个问题
无论如何修改, 删除不了拷贝base镜像的清华镜像(http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64::conda-4.9.2-py36h5fab9bb_0), 包括vim ~/.bashrc清空channel, 或者指定一个channel
猜想, 拷贝源为原有安装源
1
2
3
4
5
6
7
8
9
10
|
conda list | awk '!a[$NF]++'
# packages in environment at /home/hdp_lbg_ectech/wangke/app/anaconda3:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
aiohttp 3.8.1 pypi_0 pypi
anaconda-navigator 1.8.7 py36_0 defaults
argon2-cffi 20.1.0 py36h1d69622_2 http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
ca-certificates 2024.7.2 h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
findspark 2.0.1 pyhd8ed1ab_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
|
添加 --offline参数, -vv为debug参数
1
2
3
4
5
|
rm -rf /home/hdp_lbg_ectech/wangke/app/anaconda3/envs/pyspark3603; conda create -vv --name pyspark3603 --clone base --offline
## 这个没有参数` --no-builds`
错误变成了如下:
RuntimeError('EnforceUnusedAdapter called with url http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/_libgcc_mutex-0.1-main.tar.bz2\nThis command is using a remote connection in offline mode.\n',)
RuntimeError('EnforceUnusedAdapter called with url http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/_tflow_select-2.3.0-mkl.tar.bz2\nThis command is using a remote connection in offline mode.\n',)
|
1
2
3
4
|
rm -rf /home/hdp_lbg_ectech/wangke/app/anaconda3/envs/pyspark3603; conda create --name pyspark3603 --clone base --channel defaults
# not work
|
solution
1 导出全部包重新安装
2 删除url信息(not work)
https://blog.csdn.net/weixin_42414714/article/details/118107150 ## Error记录: RuntimeError:This command is using a remote connection in offline mode.[CondaError]-CSDN博客
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
|
# not work
find /home/hdp_lbg_ectech/wangke/app/anaconda3/lib/python3.6 -name 'urls*'
./site-packages/jedi/third_party/django-stubs/django-stubs/conf/urls
./site-packages/jedi/third_party/django-stubs/django-stubs/contrib/admindocs/urls.pyi
./site-packages/jedi/third_party/django-stubs/django-stubs/contrib/auth/urls.pyi
./site-packages/jedi/third_party/django-stubs/django-stubs/contrib/flatpages/urls.pyi
./site-packages/jedi/third_party/django-stubs/django-stubs/contrib/staticfiles/urls.pyi
./site-packages/jedi/third_party/django-stubs/django-stubs/core/checks/urls.pyi
./site-packages/jedi/third_party/django-stubs/django-stubs/urls
./site-packages/jedi/third_party/typeshed/third_party/2and3/werkzeug/urls.pyi
./site-packages/pip/_internal/utils/__pycache__/urls.cpython-36.pyc
./site-packages/pip/_internal/utils/urls.py
./site-packages/werkzeug/__pycache__/urls.cpython-36.pyc
./site-packages/werkzeug/urls.py
grep -rnw "." -e 'tsinghua'
./thulac-0.2.1.dist-info/METADATA:7:Author-email: liuzy@tsinghua.edu.cn
### bingo
### bingo
### bingo
grep -rnwI . --exclude-dir={site-packages,envs} -e "tsinghua" | more
./conda-meta/intel-openmp-2020.2-254.json:4: "channel": "http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64",
./conda-meta/intel-openmp-2020.2-254.json:58: "url": "http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/intel-openmp-2020.2-254.tar.bz2",
./conda-meta/_libgcc_mutex-0.1-main.json:4: "channel": "http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64",
./conda-meta/_libgcc_mutex-0.1-main.json:27: "url": "http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/_libgcc_mutex-0.1-main.tar.bz2",
./conda-meta/ipykernel-5.4.3-py36he448a4c_0.json:4: "channel": "http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64",
./conda-meta/ipykernel-5.4.3-py36he448a4c_0.json:1122: "url": "http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/ipykernel-5.4.3-py36he448a4c_0.tar.bz2",
./conda-meta/_tflow_select-2.3.0-mkl.json:4: "channel": "http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64",
mv conda-meta conda-meta-bak-2024-08-30
conda list
EnvironmentLocationNotFound: Not a conda environment: /home/hdp_lbg_ectech/wangke/app/anaconda3/envs/pyspark3602
!!!conda install 可以添加 --no-builds 参数
## 看来直接删除时不行的!!
替换
vim anaconda-navigator-1.8.7-py36_0.json
https://repo.anaconda.com/pkgs/main/linux-64/anaconda-navigator-1.8.7-py36_0.tar.bz2
"channel": "https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/noarch",
"url": "https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/noarch/findspark-2.0.1-pyhd8ed1ab_0.tar.bz2",
|
3 修改代码认证
4 直接拷贝整个anaconda
其实这样不好, 因为大家都在base上安装的, 但是可以保证一致性
1
2
3
4
5
6
|
tar --exclude='./envs' --exclude=**/site-packages/torch --exclude=**/site-packages/xgboost --exclude=**/site-packages/thulac -zcvf pyspark3602.tar.gz .
mkdir pyspark3602; tar -xvf pyspark3602.tar.gz -C pyspark3602 --strip-components=1
source activate pyspark3602
conda env list ## 也正常了
|
5 导出环境信息, 重新构建(推荐)
https://www.codeleading.com/article/99734905584/ fxxk
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
conda env export --no-builds | grep -v "^prefix: " > environment.yml
# 修改文件
conda config --add channels https://mirrors.ustc.edu.cn/anaconda/pkgs/free
conda config --set show_channel_urls yes
conda env create -f environment.yml
conda env create -f environment.yml -p /home/user/anaconda3/envs/env_name
ResolvePackageNotFound:
- jupyter_client=6.1.11
- prompt-toolkit=3.0.14
- findspark=2.0.1
- python_abi=3.6
- parso=0.7.1
- nodejs=12.4.0
- ipykernel=5.4.3
- pandoc=2.11.4
## this mean create failed
|
6 指定channel(not work)
https://docs.conda.io/projects/conda/en/latest/user-guide/concepts/channels.html ## Channels — conda 24.7.2.dev64 documentation
1
2
3
4
5
6
7
|
conda install scipy --channel conda-forge --channel bioconda
conda search scipy --channel file:/<path to>/local-channel --override-channels
conda create -vv --name pyspark3603 --clone base --channel conda-forge --override-channels
|
参考
