wow, so cool!
Thanks, really great tools!

yeah, it seems i will try option 2 that you mentioned.
i upload my full my configuration, and when done using google colab my configuration will gone too.

Thanks! i just prove it can run in next day, but not for the same day. i hope can run in same day too.

Syncing scheduler
Waiting for next run, sleeping for 5.13 minutes, until next sync.
Launching job: ScheduleJob(name='fetch feedback', base_task_id='', base_function=<function test_make at 0x7f91fd123d90>, queue=None, target_project='Automation/testing', single_instance=False, task_parameters={}, task_overrides={}, clone_task=True, _executed_instances=None, execution_limit_hours=None, r...

Hi @<1523701205467926528:profile|AgitatedDove14> ,
Yes i want to do that, but so far i know Task.enqueue will execute immediately, i need execute task to spesific time, and i see to do that i need scheduler and set recurring False, set time.

I tried that create scheduler, but the scheduler not created when the function executed.

the current my solution is upload my config to s3, and the pipeline will download it and read it when execute. but its decrase flexiblity.

Thanks guys, i will try to learn that first. i will updates when executing these ideas. @<1523701087100473344:profile|SuccessfulKoala55> @<1590514584836378624:profile|AmiableSeaturtle81> 🙌

I’m running the agent in ‘pip’ mode. I need to fetch certain secret values, which would be best done using Python code. If I incorporate it into the script (repository), others could deduce the path to retrieve the environment or secret value. Storing the environment variables in the clearml.config isn’t very flexible either.

hi @<1523701070390366208:profile|CostlyOstrich36> , i mean uv this None

i see, it solved right now using default_output_uri, Thanks!
i need custom output_uri for some function because split dataset and model artifacs.

my config is same like issue #763

` import clearml
from clearml import StorageManager, Dataset
from rich import print

version_clearml = clearml.version

manager = StorageManager()

print(f'clearml: {version_clearml}')
try:
minio_s3_url = 'x/x/x/x/x/x/x'
print('\n-------------download folder-------------')
target_folder = manager.download_folder(
local_folder='tmp',
remote_url=f' '
)
except Exception as e:
print(e)
print('FAILED: download fold...

nope, still looking away to set AWS S3 secret_key without doing clearml-agent init

you can spesificly use Task.add_requirements and pointing to path requirement.txt

None

Correct! Thanks AppetizingMouse58 !

remove this params will solve use_current_task=True,

still i need do this?
dataset.upload() dataset.finalize()
i have another question,
if we have uploaded data clearml, how we add data?
this is my way right now.

dataset = Dataset.create( dataset_project=metadata[2], dataset_name=metadata[3], description=description, output_uri=f" ", parent_datasets=[id_dataset_latest] )

Thanks, its works!

# downloading data from s3 manager = StorageManager() target_folder = manager.download_folder( local_folder='/tmp', remote_url=f' ` '
)

    # upload to clearml
    dataset = Dataset.create(
        dataset_project=metadata[2], 
        dataset_name=metadata[3],
        dataset_tags=tags,
        output_uri=" ` ` "
    )
    fp_target_folder = os.path.join(target_folder, minio_s3_url)
    print('>>...

Thanks for response.

from clearml import Task
from clearml.automation import TaskScheduler
from datetime import timedelta, datetime

def my_task():
    task = Task.init(...)
    # do somthinge
    print("do something")
    # sleep 10
    condition = True
    if condition:
        # i want to trigger run another task by 
        # set some config in task, but execute tomorrow/sometime
        # not directly run at the time.
        # here i use
        task_id = task.id 
        task.cl...

sorry, but can clearml pipeline do this scenario?

i see okay thanks

Hi @<1523701070390366208:profile|CostlyOstrich36> , just want to update,
this is solve by

• remove

-f 

2. change Task.force_requirements_env_freeze(False, req_path) -> Task.add_requirements(req_path)
3. change my clearml-agent settings

Hi SmugDolphin23 , i have try 1.8.4rc1, and yeah its working! Thanks!

i see thanks for the answer, i will read that reference.

it just solved. Thanks!

i see,
thanks for clarify. i just want to find other solutions to storing secret value. rightnow i just storing secret value on env in clearml.conf in my workers. but it will complicated if there is new value, i need update workers conf and redeploy workers.

it seems only happen if i change the user preference to My Works , if set Team's Work it will show like this.

my case more like there is a task/process that running but somehow its takes too long to completed. it can be because connection issue forgot to put connection timeout, a problem connection database, etc that makes status still running, but its traped in a situation like that.

so i want to force shutdown a task to failed if that happen

Hi @<1523701205467926528:profile|AgitatedDove14> , Thanks for rresponse!
this my simple code to test scheduler

import datetime
from clearml.automation import TaskScheduler
  
def test_make():
    print('test running', datetime.datetime.now())

if __name__ == '__main__':
    task_scheduler = TaskScheduler(
        sync_frequency_minutes=30,
        force_create_task_name='controller_feedback',
        force_create_task_project='Automation/Controller',
    )

    print('\n[utc_timestamp]...

clearml-agent, if you looking for clearml.conf, the place is '/root/default_clearml.conf'

it seems if i access with my dns cannot see
and if access with ip address can see