Skip to content

A programming framework for agentic AI

update

skzhang1a55ed0ed613d
diff --cc flaml/automl.py
index ddaf478e7,b959437f0..7da169abc
--- a/flaml/automl.py
+++ b/flaml/automl.py
@@@ -2072,7 -2130,7 +2132,11 @@@ class AutoML(BaseEstimator)
use_ray=None,
metric_constraints=None,
custom_hp=None,
++<<<<<<< HEAD
+ cv_score_agg_func=None,
++=======
+ skip_transform=None,
++>>>>>>> main
fit_kwargs_by_estimator=None,
**fit_kwargs,
):
@@@ -2290,39 -2350,11 +2356,47 @@@
}
```
++<<<<<<< HEAD
+ cv_score_agg_func: customized cross-validation scores aggregate function. Default to average metrics across folds. If specificed, this function needs to
+ have the following signature:
+
+ ```python
+ def cv_score_agg_func(metrics_across_folds):
+ return metric_to_minimize, metrics_to_log
+ ```
+ The input "metrics_across_folds" is a list of 2-tuples. Each tuple records the loss and metrics information of the corresponding fold.
+ On each tuple, the first element is a float number that represents the loss score to minimize, and the second is a dict of all the metrics to log or None.
+ It returns the final aggregate result of all folds. A float number of the minimization objective, and a dictionary as the metrics to log or None.
+ E.g.,
+
+ ```python
+ def cv_score_agg_func(metrics_across_folds):
+ metric_to_minimize = sum([tem[0] for tem in metrics_across_folds])/len(metrics_across_folds)
+ metrics_to_log = None
+ for single_fold in metrics_across_folds:
+ if single_fold[1] is None:
+ break
+ elif metrics_to_log is None:
+ metrics_to_log = single_fold[1]
+ else:
+ metrics_to_log = {k: metrics_to_log[k] + v for k, v in single_fold[1].items()}
+ if metrics_to_log:
+ n = len(metrics_across_folds)
+ metrics_to_log = {k: v / n for k, v in metrics_to_log.items()}
+ return metric_to_minimize, metrics_to_log
+ ```
+
+ fit_kwargs_by_estimator: dict, default=None | The user specified keywords arguments, grouped by estimator name.
+ For TransformersEstimator, available fit_kwargs can be found from
+ [TrainingArgumentsForAuto](nlp/huggingface/training_args).
+ e.g.,
++=======
+ skip_transform: boolean, default=False | Whether to pre-process data prior to modeling.
+ fit_kwargs_by_estimator: dict, default=None | The user specified keywords arguments, grouped by estimator name.
+ For TransformersEstimator, available fit_kwargs can be found from
+ [TrainingArgumentsForAuto](nlp/huggingface/training_args).
+ e.g.,
++>>>>>>> main
```python
fit_kwargs_by_estimator = {