Download toolkit git clone https://github.com/mozilla/TTS.git Install and config env conda create -n convlab2 python=3.6.2 cd ConvLab-2 install cpu version of pytorch (run on personal computer)conda install pytorch==1.5.1 torchvision==0.6.1 cpuonly -c pytorch otherwise, auto installation cannot find package version. pip install -e .
Configuring launch.json in vscode "configurations": [ { "name": "Python: my tod training", "type": "python", "request": "launch", "program": "${file}", "console": "integratedTerminal", "args": [ "--task=usdl", "--model_type=bert", "--model_name_or_path=bert-base-uncased", "--output_dir=", "--do_train", "--do_eval", "--mlm", "--do_lower_case", "--evaluate_during_training", "--save_steps=2500", "--logging_steps=1000", "--per_gpu_train_batch_size=1", "--per_gpu_eval_batch_size=1", "--only_last_turn" ] } ] Running Debug Session open my_tod_pretraining.py and press F5 to start a debug session using the configuration shown above. bug report: Debugging adding breakpoint on line 458 of my_tod_pretraining.py, and use F11:Step into to dig into the situation....
ToD-BERT the Paper 数据集 不同的数据集可以帮助模型达到不同的熟练效果 MetaLWOZ 预测数据 … ToD-BERT怎么训练的？ mlm contrastive function 两者都有误差，因此才可以被训练。 空间结构能够捕获差异，发现细微结构。 在服务器上运行ToD-BERT训练 进入服务器，激活环境source activate todbert_env 进入/media/HD1/dche/ToD-BERT文件夹cd /media/HD1/dche/ToD-BERT 查看GPU资源占用情况nvidia-smi，然后选择目前占用情况较低的一张GPU进行训练即可 运行训练shell脚本文件CUDA_VISIBLE_DEVICES=0 ./run_tod_lm_pretraining.sh 0 bert bert-base-uncased save/pretrain/ToD-BERT-MLM --only_last_turn --data_path ./../dialog_datasets。根据第三步选择的几号卡，就把对应的0改成几，此处默认单卡训练。如果一切正常的话，再读入数据集数据后，就会开始训练了，有进度条出现就Ok了。常见的没跑起来的情况是CUDA out of memory。 ToD-BERT本地调用 将ToD-BERT模型下载至本地 包含ToD-BERT所需的python包，并定义模型路径 import torch from transformers import * BERT = <path_to_the_downloaded_tod-bert> # 注意此处的路径要使用从根目录开始的绝对路径，而非从用户~目录开始的相对路径。 model_class, tokenizer_class, config_class = BertModel, BertTokenizer, BertConfig tokenizer = tokenizer_class.from_pretrained(BERT) tod_bert = model_class.from_pretrained(BERT) 使用ToD-BERT文档中的示例 # Encode text input_text = "[CLS] [SYS] Hello, what can I help with you today?...