Hi @<1523701070390366208:profile|CostlyOstrich36> ,
i use vpn and set location in US still cannot access too.
when i set location to Germany, it can access.
any idea to solve this from user side?
Hi @<1523701994743664640:profile|AppetizingMouse58> , i have update on #228. Thanks!
I’m running the agent in ‘pip’ mode. I need to fetch certain secret values, which would be best done using Python code. If I incorporate it into the script (repository), others could deduce the path to retrieve the environment or secret value. Storing the environment variables in the clearml.config isn’t very flexible either.
yup,
example just need choosing between SGD, Adam, AdamW on optimizer field
I see, yeah my alternative solution right now is just to show the list of options outside on ClearML UI.
# downloading data from s3 manager = StorageManager() target_folder = manager.download_folder( local_folder='/tmp', remote_url=f'
` '
)
# upload to clearml
dataset = Dataset.create(
dataset_project=metadata[2],
dataset_name=metadata[3],
dataset_tags=tags,
output_uri=" ` ` "
)
fp_target_folder = os.path.join(target_folder, minio_s3_url)
print('>>...
still i need do this?dataset.upload() dataset.finalize()
i have another question,
if we have uploaded data clearml, how we add data?
this is my way right now.
dataset = Dataset.create( dataset_project=metadata[2], dataset_name=metadata[3], description=description, output_uri=f"
", parent_datasets=[id_dataset_latest] )
yup correct. but the scheduler not created idk why. here my code and the log
from doctest import Example
from clearml.automation import TriggerScheduler, TaskScheduler
from clearml import Task
import json
def open_json(fp):
with open(fp, 'r') as f:
my_dictionary = json.load(f)
return my_dictionary
def trigger_task_func(task_id):
print("trigger running...")
try:
previous_task = Task.get_task(task_id=task_id)
print(previous_task.artifact...
hi i have similar case, but can we scheduled new task here?
def trigger_task_func(task_id):
print("trigger running...")
try:
previous_task = Task.get_task(task_id=task_id)
print(previous_task.artifacts)
try:
fp = previous_task.artifacts['latest_condition'].get_local_copy()
params = open_json(fp)
last_index = params.get('last_index')
day_n = params.get('iteration')
print("Success Fetching", param...
Hi @<1523701205467926528:profile|AgitatedDove14> ,
Yes i want to do that, but so far i know Task.enqueue will execute immediately, i need execute task to spesific time, and i see to do that i need scheduler and set recurring False, set time.
I tried that create scheduler, but the scheduler not created when the function executed.
Thanks @<1523701205467926528:profile|AgitatedDove14> , right now i just use trigger to send notification and do it manually. ClearML Superb!
my config is same like issue #763
` import clearml
from clearml import StorageManager, Dataset
from rich import print
version_clearml = clearml.version
manager = StorageManager()
print(f'clearml: {version_clearml}')
try:
minio_s3_url = 'x/x/x/x/x/x/x'
print('\n-------------download folder-------------')
target_folder = manager.download_folder(
local_folder='tmp',
remote_url=f' '
)
except Exception as e:
print(e)
print('FAILED: download fold...
maybe accidently install my custom solution on this https://github.com/muhammadAgfian96/clearml/commit/01db9aa40537a6c2f83977220423556a48614c3a at that time. so i said the test is passed.
https://github.com/mert-kurttutan/torchview
maybe can try this one, and can send to logger clearml at the end.
i set like this: for init Task Scheduler
task_scheduler = TaskScheduler(
sync_frequency_minutes=5,
force_create_task_name='controller_feedback',
force_create_task_project='Automation/Controller',
)
i running this on 2.35 am, but the job not launching after 2.40 am
oh okay, so i need to set that to path ssd, yeah?
is it this one? or there is
docker_internal_mounts {
sdk_cache: "/clearml_agent_cache"
apt_cache: "path/to/ssd/apt-cache"
ssh_folder: "/root/.ssh"
pip_cache: "path/to/ssd/clearml-cache/pip"
poetry_cache: "/mnt/hdd_2/clearml-cache/pypoetry"
vcs_cache: "path/to/ssd/clearml-cache/vcs-cache"
venv_build: "path/to/ssd/clearml-cache/venvs-builds"
pip_download: "path/to/ssd/cle...
Hi @<1523701070390366208:profile|CostlyOstrich36>
i attach for complete log
here my structure:
.
├── app
│ ├── backend
│ └── frontend
├── assets
│ ├── demo-app-sample.png
│ └── workflow.png
├── config
│ ├── clearml.conf
│ ├── list_models.py
│ ├── list_optimizer.py
│ ├── __pycache__
│ └── train_config.py
├── docker
│ ├── Dockerfile
│ ├── Dockerfile.app
│ ├── requirements.prod.txt
│ ├── requirements.train.txt
│ └── requirements.txt
├── lightning_logs
├── Mak...
from src.net import Classifier
ModuleNotFoundError: No module named 'src'
hmm yeah i think, this is not possible to share a whole script here.
i attach train.py here,
and to run it i do python src/train.py
Hi @<1523701070390366208:profile|CostlyOstrich36> , just want to update,
this is solve by
- remove
-f
- change Task.force_requirements_env_freeze(False, req_path) -> Task.add_requirements(req_path)
- change my clearml-agent settings
alright, will try, i just worried about if execution mode is docker mode? should i mount to /var/run/docker.sock?
hi @<1523701070390366208:profile|CostlyOstrich36> , i mean uv this None
Correct! Thanks AppetizingMouse58 !
Hi AgitatedDove14 ,
right now i can delete, i set configuration to see hidden project/file on UI, and it show task related (e.g pipeline, dataset) on that project. but yeah, every time i make project with subproject inside, there is [undefined] there. i create project using code and UI, the result is same.
this is my docker-compose conf. i change all port 8XXX to 7XXX nad also change /opt/cleaml to /mnt/hdd_2/clearml.
` version: "3.6"
services:
apiserver:
command:
- apiserver
...
Hi AppetizingMouse58 , this is when in Team's Work View :
`
Payload
{"id":["75d04598197a445ebef533814022c58d"],"include_stats":true,"check_own_contents":true,"search_hidden":true}
Response
{"meta":{"id":"c4ee9cb1c4594040bd0b44499d5e9970","trx":"c4ee9cb1c4594040bd0b44499d5e9970","endpoint":{"name":"projects.get_all_ex","requested_version":"2.20","actual_version":"1.0"},"result_code":200,"result_subcode":0,"result_msg":"OK","error_stack":"","error_data":{}},"data":{"projects":[{"id":"7...
` {"meta":{"id":"369cbeafb3ec4b36ae7c07e781be5941","trx":"369cbeafb3ec4b36ae7c07e781be5941","endpoint":{"name":"projects.get_all_ex","requested_version":"2.20","actual_version":"1.0"},"result_code":200,"result_subcode":0,"result_msg":"OK","error_stack":"","error_data":{}},"data":{"projects":[{"company":{"id":"d1bd92a3b039400cbafc60a7a5b1e52b"},"created":"2022-09-26T01:33:28.216000+00:00","id":"5664bd88f48d4868a80805f6d824d5a9","name":"ex-1/sub-ex-1","user":{"id":"a174c4e36b0446a7b3b5dd1ff5261...
it seems only happen if i change the user preference to My Works , if set Team's Work it will show like this.