Reputation
Badges 1
69 × Eureka!Or do I have to dive into the code in train function and write the code there?
Hi SuccessfulKoala55 ,
First I create the task and change the output_uri to some folder I createdtask = Task.init(project_name='main', task_name='task', output_uri=r'C:\Users\Chen\Desktop\folder)
Than I used:task.upload_artifact
In the init I passed output_uri=Folder
But now I have diffrent problem, in the dataset.get_mutable_local_copy
I get:
File "/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/datasets/dataset.py", line 1276, in _download_part
raise ValueError("Could not download dataset id={} entry={}".format(self._id, data_artifact_name))
ValueError: Could not download dataset
Maybe I should pass something in extra_docker_arguments?
CostlyOstrich36
The sdk.aws.s3.credentials.0.host and sdk.aws.s3.credentials.0.key, yes
Now, in other task I want to upload the artifact by using get_local_copy()dataset_upload_task = Task.get_task(task_id=args['dataset_task_id']) local_json = dataset_upload_task.artifacts['dataset'].get_local_copy()
But the path that I got contain '' and '/' (i.e combination of unix and win foramt)
SuccessfulKoala55
Hi,
Using task.upload_artifact
CostlyOstrich36 thanks.
Maybe I should pass something in extra_docker_arguments in the config file?
I just added -p port_num:port_num to extra_docker_arguments
CostlyOstrich36 When I do
logger = task.get_logger()
logger.report_scalar(title='evaluate', series='score', value=5, iteration=task.get_last_iteration())
train(model_dir=trained_model_dst, pipeline_config_path=pipeline_config_path, save_checkpoints_steps=args.checkpoints)
It only captures the first iteration...
CostlyOstrich36 having the reported tensorflow scalars show up on ClearML
And again only in docker mode CostlyOstrich36
I found the problem I think, thanks!
Get stuck on:
Collecting pip<20.2
Using cached pip-20.1.1-py2.py3-none-any.whl (1.5 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 21.3.1
Uninstalling pip-21.3.1:
Successfully uninstalled pip-21.3.1
Successfully installed pip-20.1.1
Collecting Cython
Using cached Cython-0.29.24-cp38-cp38-win_amd64.whl (1.7 MB)
Installing collected packages: Cython
Successfully installed Cython-0.29.24
Collecting boto3==1.19.9
Using cach...
Haa,
How can I change it?
The pipline is on services
and the first task on defult
ExasperatedCrab78
pipe = PipelineController( name='grab2train_tf', project='main pipeline', version='0.0.1', abort_on_failure=True, # any failed step will cause the pipeline to immediately abort, stop all running steps, and mark the pipeline as failed. )
I'm missing something? CostlyOstrich36
MelancholyElk85 How do I change the SCRIPT PATH/SCRIPT PATH