A programming framework for agentic AI
update doc about scheduler exception (#564)
Chi Wang1af682b7f5f3
diff --git a/flaml/tune/tune.py b/flaml/tune/tune.pyindex a4f55fb3a..ce123e9a9 100644--- a/flaml/tune/tune.py+++ b/flaml/tune/tune.py@@ -83,6 +83,8 @@ def report(_metric=None, **kwargs):Raises:StopIteration (when not using ray, i.e., _use_ray=False):A StopIteration exception is raised if the trial has been signaled to stop.+ SystemExit (when using ray):+ A SystemExit exception is raised if the trial has been signaled to stop by ray."""global _use_rayglobal _verbose@@ -239,9 +241,11 @@ def run(respectively. You can also provide a self-defined scheduler instanceof the TrialScheduler class. When 'asha' or self-defined scheduler isused, you usually need to report intermediate results in the evaluation- function via 'tune.report()'. In addition, when 'use_ray' is not enabled,- you also need to stop the evaluation function by explicitly catching the- `StopIteration` exception, as shown in the following example.+ function via 'tune.report()'.+ If you would like to do some cleanup opearation when the trial is stopped+ by the scheduler, you can catch the `StopIteration` (when not using ray)+ or `SystemExit` (when using ray) exception explicitly,+ as shown in the following example.Please find more examples using different types of schedulersand how to set up the corresponding evaluation functions intest/tune/test_scheduler.py, and test/tune/example_scheduler.py.@@ -252,7 +256,8 @@ def run(intermediate_score = evaluation_fn(step, width, height)try:tune.report(iterations=step, mean_loss=intermediate_score)- except StopIteration:+ except (StopIteration, SystemExit):+ # do cleanup operation herereturn```search_alg: An instance of BlendSearch as the search algorithmdiff --git a/test/automl/test_classification.py b/test/automl/test_classification.pyindex b0dfd6b55..69cd019f4 100644--- a/test/automl/test_classification.py+++ b/test/automl/test_classification.py@@ -99,7 +99,6 @@ class TestClassification(unittest.TestCase):"ensemble": True,}automl.fit(X, y, **automl_settings)- assert automl.model is not Noneautoml = AutoML()try:diff --git a/website/docs/Use-Cases/Tune-User-Defined-Function.md b/website/docs/Use-Cases/Tune-User-Defined-Function.mdindex b46df63b8..f55b37a7d 100644--- a/website/docs/Use-Cases/Tune-User-Defined-Function.md+++ b/website/docs/Use-Cases/Tune-User-Defined-Function.md@@ -353,7 +353,7 @@ tune.run(.., scheduler=my_scheduler, ...)- Different from the case when the `flaml` scheduler is used, the amount of resources to use at each iteration is not suggested by the search algorithm through the `resource_attr` in a configuration. You need to specify the evaluation schedule explicitly by yourself in the `evaluation_function` and **report intermediate results (using `tune.report()`) accordingly**. In the following code example, we use the ASHA scheduler by setting `scheduler="asha"`. We specify `resource_attr`, `min_resource`, `min_resource` and `reduction_factor` the same way as in the previous example (when "flaml" is used as the scheduler). We perform the evaluation in a customized schedule.-- Use ray backend or not? You can choose to use ray backend or not by specifying `use_ray=True` or `use_ray=False`. When ray backend is not used, i.e., `use_ray=False`, you also need to stop the evaluation function by explicitly catching the `StopIteration` exception, as shown in the last two lines of the evaluation function `obj_w_intermediate_report()` in the following code example.+- Use ray backend or not? You can choose to use ray backend or not by specifying `use_ray=True` or `use_ray=False`. When ray backend is not used, i.e., `use_ray=False`, you also need to stop the evaluation function by explicitly catching the `StopIteration` exception, as shown in the end of the evaluation function `obj_w_intermediate_report()` in the following code example.```pythondef obj_w_intermediate_report(resource_attr, X_train, X_test, y_train, y_test, min_resource, max_resource, config):@@ -375,7 +375,8 @@ def obj_w_intermediate_report(resource_attr, X_train, X_test, y_train, y_test, m# need to report the resource attribute used and the corresponding intermediate resultstry:tune.report(sample_size=resource, loss=test_loss)- except StopIteration:+ except (StopIteration, SystemExit):+ # do cleanup operation herereturnresource_attr = "sample_size"@@ -399,6 +400,9 @@ analysis = tune.run()```+- If you would like to do some cleanup opearation when the trial is stopped+by the scheduler, you can do it when you catch the `StopIteration` (when not using ray) or `SystemExit` (when using ray) exception explicitly.+### Warm startRelated arguments: