conda-env-clone-install-create-error-with-mirror-custom-channel 2024-08-30 unsort unsort [TOC] 问题原因 突然不能克隆base环境了 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 rm-rfpyspark3602;condacreate--name pyspark3602 --clone base Source:/home/hdp_lbg_ectech/wangke/app/anaconda3Destination:/home/hdp_lbg_ectech/wangke/app/anaconda3/envs/pyspark3602Thefollowingpackagescannotbeclonedoutoftherootenvironment:-http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64::conda-4.9.2-py36h5fab9bb_0-defaults/linux-64::conda-build-3.10.5-py36_0Packages:195Files:64657...CondaHTTPError:HTTP404NOTFOUNDforurl<http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/keras-applications-1.0.8-py_1.tar.bz2>Elapsed:00:00.351600AnHTTPerroroccurredwhentryingtoretrievethisURL.HTTPerrorsareoftenintermittent,andasimpleretrywillgetyouonyourway. 第一个问题 这里的主要问题是使用了清华的镜像. 清华镜像只有文件. ...
save DataFrame as partition of hive table 2023-04-27 data data 使用DataFrame直接创建hive表, 并作为其中的一个分区数据 test 1 1 2 3 4 5 6 table .write .format("hive") .mode("overwrite") .option("path", inputPath + "_table") .insertInto(tableName) error, 需要先创建表 1 Exception in thread "main" org.apache.spark.sql.AnalysisException: Table not found: hdp_lbg_ectech_ads.zp_compensate_ad_detail_test1; test 2 首 ...
ssh git github permission denied problem 2022-11-03 unsort git github ssh 问题描述 原来在/etc/hosts配置了github的ip解析, 突然有一天push很慢, 甚至经常timeout. 然后自己把hosts中关于g ...
pyspark udf udaf with jar 2022-10-21 unsort unsort 问题描述 使用scala开发了udaf, 在scala程序中能使用, 无法在pyspark中使用 使用udaf有两种方法: 第一种是hive使用 1 2 3 ...
dropout笔记 2021-04-27 tf dl tf dl 原理 dropout原理, 随机丢弃一些(输入)神经元, 防止参数过拟合 Applies Dropout to the input. Dropout consists in randomly setting a fraction rate of input units to 0 at each update during training time, which helps prevent overfitting. The units that are kept are scaled by 1 ...
