Skip to content

Campaign smoke

Campaign smoke: ten datasets at matched knobs

The internal quality campaign (scripts/compare.py, campaign knobs, best variant per library, latest run per dataset):

dataset metric bonsai xgboost lightgbm catboost best
f_electricity auc 0.9458 0.9353 0.9441 0.9178 bonsai
f_elevators rmse 0.0023 0.0024 0.0023 0.0026 bonsai
f_house_sales rmse 131655.8118 121828.8445 122212.9584 124094.5336 xgboost
f_letter acc 0.9613 0.9240 0.9610 0.9223 bonsai
f_magic_telescope auc 0.9360 0.9330 0.9352 0.9335 bonsai
f_phoneme auc 0.9435 0.9283 0.9379 0.9396 bonsai
f_satimage acc 0.9191 0.9012 0.9191 0.9059 bonsai
f_superconduct rmse 10.3519 10.7637 10.7227 12.1154 bonsai
f_wine_quality rmse 0.6335 0.6385 0.6386 0.6505 bonsai
g_electricity auc 0.9482 0.9353 0.9441 0.9178 bonsai
g_elevators rmse 0.0023 0.0024 0.0023 0.0026 bonsai
g_house_sales rmse 125913.9209 121828.8445 122212.9584 124094.5336 xgboost
g_letter acc 0.9613 0.9240 0.9610 0.9223 bonsai
g_magic_telescope auc 0.9352 0.9330 0.9352 0.9335 lightgbm
g_phoneme auc 0.9401 0.9283 0.9379 0.9396 bonsai
g_satimage acc 0.9191 0.9012 0.9191 0.9059 bonsai
g_superconduct rmse 10.3959 10.7637 10.7227 12.1154 bonsai
g_wine_quality rmse 0.6373 0.6385 0.6386 0.6505 bonsai
h_electricity auc 0.9482 0.9353 0.9441 0.9178 bonsai
h_house_sales rmse 126530.7992 121828.8445 122212.9584 124094.5336 xgboost
h_magic_telescope auc 0.9349 0.9330 0.9352 0.9335 lightgbm
h_phoneme auc 0.9392 0.9283 0.9379 0.9396 catboost
h_superconduct rmse 10.4083 10.7637 10.7227 12.1154 bonsai
h_wine_quality rmse 0.6374 0.6385 0.6386 0.6505 bonsai
i_electricity auc 0.9457 0.9353 0.9441 0.9178 bonsai
i_elevators rmse 0.0023 0.0024 0.0023 0.0026 bonsai
i_house_sales rmse 126447.7670 121828.8445 122212.9584 124094.5336 xgboost
i_letter acc 0.9613 0.9240 0.9610 0.9223 bonsai
i_magic_telescope auc 0.9356 0.9330 0.9352 0.9335 bonsai
i_phoneme auc 0.9415 0.9283 0.9379 0.9396 bonsai
i_satimage acc 0.9191 0.9012 0.9191 0.9059 bonsai
i_superconduct rmse 10.3959 10.7637 10.7227 12.1154 bonsai
i_wine_quality rmse 0.6373 0.6385 0.6386 0.6505 bonsai
j_bike_sharing rmse 39.9162 53.8279 49.2892 45.0404 bonsai
p63_house_mb1023 rmse 121942.8335 122749.8611 122570.7576 124094.5336 bonsai
p63_house_mb511 rmse 121958.5113 119517.0121 121831.7427 124094.5336 xgboost
p_elev_obl_h0 rmse 0.0026 0.0024 0.0023 0.0026 lightgbm
p_house_bins rmse 118305.3912 122749.8611 122570.7576 124094.5336 bonsai
p_house_depth rmse 132789.1247 121067.0042 121998.2617 121420.6540 xgboost
p_house_iters rmse 127481.7850 117281.9561 117400.1156 115253.0073 catboost
p_house_leaf rmse 134131.8513 121521.4630 124794.2564 124094.5336 xgboost
p_letter_iters acc 0.9613 0.9415 0.9683 0.0387 lightgbm
p_letter_lr2 acc 0.9620 0.9397 0.9673 0.0387 lightgbm
p_super_obl_h0 rmse 11.8523 10.7637 10.7227 12.1154 lightgbm
q_bike_sharing_mse rmse 41.1407 44.4491 42.0133 48.5028 bonsai
q_electricity auc 0.9481 0.9353 0.9441 0.8262 bonsai
q_elevators rmse 0.0023 0.0024 0.0023 0.0026 bonsai
q_elevators_huber rmse 0.0023 0.0024 0.0023 0.0026 lightgbm
q_elevators_mae rmse 0.0027 0.0029 0.0028 0.0031 bonsai
q_elevators_quantile rmse 0.0027 0.0028 0.0028 0.0031 bonsai
q_house_sales rmse 130746.3931 121828.8445 122212.9584 124094.5336 lightgbm (goss)
q_house_sales_huber rmse 140605.7902 17289465332.0396 373651.6657 622692.1388 bonsai
q_house_sales_mae rmse 135709.1681 138279.3358 139079.4959 143292.1296 bonsai
q_house_sales_quantile rmse 137926.7874 136812.5461 137758.5501 143292.1296 xgboost
q_letter acc 0.9555 0.9240 0.9610 0.0387 lightgbm (goss)
q_phoneme auc 0.9435 0.9283 0.9379 0.8417 bonsai
q_satimage acc 0.9199 0.9012 0.9191 0.1851 bonsai
q_smoke_mae rmse 0.6833 0.6877 0.6859 0.6844 bonsai
q_superconduct rmse 10.3668 10.7637 10.7227 12.1154 bonsai
q_superconduct_huber rmse 11.2496 12.8570 27.5366 14.0227 bonsai
q_superconduct_mae rmse 11.6526 12.1975 11.9220 13.4911 bonsai
q_superconduct_quantile rmse 11.6092 12.1112 12.0634 13.4911 bonsai
q_wine_quality rmse 0.6315 0.6385 0.6386 0.6505 bonsai
q_wine_quality_huber rmse 0.6317 170.0988 0.6406 12.9831 bonsai
q_wine_quality_mae rmse 0.6833 0.6877 0.6859 0.6844 bonsai
q_wine_quality_quantile rmse 0.7081 0.6897 0.7761 0.6844 catboost
v_elevators_60 rmse 0.0026 0.0024 0.0023 0.0026 lightgbm
v_elevators_61 rmse 0.0023 0.0024 0.0023 0.0026 bonsai
v_house_sales_60 rmse 137968.7153 121828.8445 122212.9584 124094.5336 xgboost
v_house_sales_61 rmse 131841.3646 121828.8445 122212.9584 124094.5336 xgboost
v_letter_59 acc 0.9515 0.9240 0.9610 0.9223 lightgbm
v_letter_62 acc 0.9613 0.9240 0.9610 0.9223 bonsai
v_satimage_62 acc 0.9191 0.9012 0.9191 0.9059 bonsai
v_superconduct_60 rmse 11.8523 10.7637 10.7227 12.1154 lightgbm
v_superconduct_61 rmse 10.3519 10.7637 10.7227 12.1154 bonsai
v_wine_quality_61 rmse 0.6335 0.6385 0.6386 0.6505 bonsai

Source: quality-campaign-2026-07.jsonl. Aggregate record; narrative in benchmarks/quality-campaign-2026-07.md, decisions 56 and 57.