ML之回归预测：利用两种机器学习算法(LiR，XGBoost(调优+重要性可视化+特征选择模型))对无人驾驶汽车系统参数(2017年的data,18+2)进行回归预测值VS真实值

​输出结果​

输出结果

1、LiR模型

LiR：The value of default measurement of LiR is 0.8729775261968014

LiR：R-squared value of DecisionTreeRegressor: 0.8729775261968014

2、XGBoost模型

T1、调用XGBR_GSCV_Shuffle()函数，调优+重要性可视化+特征选择模型

XGBR_model = XGBRegressor( learning_rate=0.06, max_depth= 4, n_estimators=100 )    #XGBR_GSCV_Shuffle()函数，第一次得到最佳参数组合，输出准确度： 0.9312586298921468

XGBR_model = XGBRegressor( learning_rate=0.15, max_depth= 4, n_estimators=100 )    #XGBR_GSCV_Shuffle()函数，第二次得到最佳参数组合，输出准确度： 0.9361222829659452

XGBR_model = XGBRegressor( learning_rate=0.03, max_depth= 5, n_estimators=200 )    #XGBR_GSCV_Shuffle()函数，第三次得到最佳参数组合，输出准确度： 0.9335316602435876

`Best: -7.044124 using {'learning_rate': 0.03, 'max_depth': 5, 'n_estimators': 200}XGBR_GSCV_Shuffle score: 2.65407690407428-13.263615 (3.437888) with: {'learning_rate': 0.03, 'max_depth': 4, 'n_estimators': 100}-7.085101 (5.544846) with: {'learning_rate': 0.03, 'max_depth': 4, 'n_estimators': 200}-13.266334 (3.458229) with: {'learning_rate': 0.03, 'max_depth': 5, 'n_estimators': 100}-7.044124 (5.800332) with: {'learning_rate': 0.03, 'max_depth': 5, 'n_estimators': 200}-13.379665 (3.515279) with: {'learning_rate': 0.03, 'max_depth': 6, 'n_estimators': 100}-7.185696 (5.878527) with: {'learning_rate': 0.03, 'max_depth': 6, 'n_estimators': 200}-13.479146 (3.629065) with: {'learning_rate': 0.03, 'max_depth': 7, 'n_estimators': 100}-7.324944 (5.854973) with: {'learning_rate': 0.03, 'max_depth': 7, 'n_estimators': 200}-7.143094 (5.637506) with: {'learning_rate': 0.06, 'max_depth': 4, 'n_estimators': 100}-7.593377 (6.216784) with: {'learning_rate': 0.06, 'max_depth': 4, 'n_estimators': 200}-7.098928 (5.750214) with: {'learning_rate': 0.06, 'max_depth': 5, 'n_estimators': 100}-7.597613 (6.403983) with: {'learning_rate': 0.06, 'max_depth': 5, 'n_estimators': 200}-7.210929 (5.854905) with: {'learning_rate': 0.06, 'max_depth': 6, 'n_estimators': 100}-7.759291 (6.498452) with: {'learning_rate': 0.06, 'max_depth': 6, 'n_estimators': 200}-7.348396 (5.867050) with: {'learning_rate': 0.06, 'max_depth': 7, 'n_estimators': 100}-7.914092 (6.526464) with: {'learning_rate': 0.06, 'max_depth': 7, 'n_estimators': 200}-7.514619 (6.090178) with: {'learning_rate': 0.09, 'max_depth': 4, 'n_estimators': 100}-7.792390 (6.343156) with: {'learning_rate': 0.09, 'max_depth': 4, 'n_estimators': 200}-7.506378 (6.281410) with: {'learning_rate': 0.09, 'max_depth': 5, 'n_estimators': 100}-7.757921 (6.477667) with: {'learning_rate': 0.09, 'max_depth': 5, 'n_estimators': 200}-7.626987 (6.321250) with: {'learning_rate': 0.09, 'max_depth': 6, 'n_estimators': 100}-7.830667 (6.496410) with: {'learning_rate': 0.09, 'max_depth': 6, 'n_estimators': 200}-7.873006 (6.432751) with: {'learning_rate': 0.09, 'max_depth': 7, 'n_estimators': 100}-8.036536 (6.584526) with: {'learning_rate': 0.09, 'max_depth': 7, 'n_estimators': 200}-7.672704 (6.222572) with: {'learning_rate': 0.12000000000000001, 'max_depth': 4, 'n_estimators': 100}-7.916448 (6.418164) with: {'learning_rate': 0.12000000000000001, 'max_depth': 4, 'n_estimators': 200}-7.724868 (6.419296) with: {'learning_rate': 0.12000000000000001, 'max_depth': 5, 'n_estimators': 100}-7.893062 (6.541605) with: {'learning_rate': 0.12000000000000001, 'max_depth': 5, 'n_estimators': 200}-7.849538 (6.506693) with: {'learning_rate': 0.12000000000000001, 'max_depth': 6, 'n_estimators': 100}-7.949133 (6.580036) with: {'learning_rate': 0.12000000000000001, 'max_depth': 6, 'n_estimators': 200}-8.021275 (6.522834) with: {'learning_rate': 0.12000000000000001, 'max_depth': 7, 'n_estimators': 100}-8.115124 (6.590436) with: {'learning_rate': 0.12000000000000001, 'max_depth': 7, 'n_estimators': 200}-7.851446 (6.380559) with: {'learning_rate': 0.15000000000000002, 'max_depth': 4, 'n_estimators': 100}-7.962357 (6.452926) with: {'learning_rate': 0.15000000000000002, 'max_depth': 4, 'n_estimators': 200}-7.752629 (6.526754) with: {'learning_rate': 0.15000000000000002, 'max_depth': 5, 'n_estimators': 100}-7.870802 (6.591447) with: {'learning_rate': 0.15000000000000002, 'max_depth': 5, 'n_estimators': 200}-7.828501 (6.482895) with: {'learning_rate': 0.15000000000000002, 'max_depth': 6, 'n_estimators': 100}-7.892413 (6.509397) with: {'learning_rate': 0.15000000000000002, 'max_depth': 6, 'n_estimators': 200}-8.141324 (6.636931) with: {'learning_rate': 0.15000000000000002, 'max_depth': 7, 'n_estimators': 100}-8.182099 (6.635204) with: {'learning_rate': 0.15000000000000002, 'max_depth': 7, 'n_estimators': 200}-7.938719 (6.490107) with: {'learning_rate': 0.18000000000000002, 'max_depth': 4, 'n_estimators': 100}-8.017980 (6.506082) with: {'learning_rate': 0.18000000000000002, 'max_depth': 4, 'n_estimators': 200}-7.938695 (6.610782) with: {'learning_rate': 0.18000000000000002, 'max_depth': 5, 'n_estimators': 100}-8.012643 (6.660132) with: {'learning_rate': 0.18000000000000002, 'max_depth': 5, 'n_estimators': 200}-8.011816 (6.616109) with: {'learning_rate': 0.18000000000000002, 'max_depth': 6, 'n_estimators': 100}-8.052129 (6.641090) with: {'learning_rate': 0.18000000000000002, 'max_depth': 6, 'n_estimators': 200}-8.118405 (6.560621) with: {'learning_rate': 0.18000000000000002, 'max_depth': 7, 'n_estimators': 100}-8.131590 (6.550569) with: {'learning_rate': 0.18000000000000002, 'max_depth': 7, 'n_estimators': 200}-7.915589 (6.338897) with: {'learning_rate': 0.21000000000000002, 'max_depth': 4, 'n_estimators': 100}-8.019436 (6.383854) with: {'learning_rate': 0.21000000000000002, 'max_depth': 4, 'n_estimators': 200}-7.956674 (6.487618) with: {'learning_rate': 0.21000000000000002, 'max_depth': 5, 'n_estimators': 100}-8.028267 (6.514906) with: {'learning_rate': 0.21000000000000002, 'max_depth': 5, 'n_estimators': 200}-8.036983 (6.583115) with: {'learning_rate': 0.21000000000000002, 'max_depth': 6, 'n_estimators': 100}-8.085323 (6.596389) with: {'learning_rate': 0.21000000000000002, 'max_depth': 6, 'n_estimators': 200}-8.254193 (6.565100) with: {'learning_rate': 0.21000000000000002, 'max_depth': 7, 'n_estimators': 100}-8.269231 (6.561241) with: {'learning_rate': 0.21000000000000002, 'max_depth': 7, 'n_estimators': 200}-8.143765 (6.593441) with: {'learning_rate': 0.24000000000000002, 'max_depth': 4, 'n_estimators': 100}-8.218321 (6.600359) with: {'learning_rate': 0.24000000000000002, 'max_depth': 4, 'n_estimators': 200}-8.191637 (6.690425) with: {'learning_rate': 0.24000000000000002, 'max_depth': 5, 'n_estimators': 100}-8.222861 (6.676068) with: {'learning_rate': 0.24000000000000002, 'max_depth': 5, 'n_estimators': 200}-8.230726 (6.661499) with: {'learning_rate': 0.24000000000000002, 'max_depth': 6, 'n_estimators': 100}-8.260381 (6.658228) with: {'learning_rate': 0.24000000000000002, 'max_depth': 6, 'n_estimators': 200}-8.470876 (6.728413) with: {'learning_rate': 0.24000000000000002, 'max_depth': 7, 'n_estimators': 100}-8.480391 (6.731554) with: {'learning_rate': 0.24000000000000002, 'max_depth': 7, 'n_estimators': 200}-7.967612 (6.650860) with: {'learning_rate': 0.27, 'max_depth': 4, 'n_estimators': 100}-8.051922 (6.656565) with: {'learning_rate': 0.27, 'max_depth': 4, 'n_estimators': 200}-8.121717 (6.575363) with: {'learning_rate': 0.27, 'max_depth': 5, 'n_estimators': 100}-8.160381 (6.577406) with: {'learning_rate': 0.27, 'max_depth': 5, 'n_estimators': 200}-8.070251 (6.545575) with: {'learning_rate': 0.27, 'max_depth': 6, 'n_estimators': 100}-8.099537 (6.546567) with: {'learning_rate': 0.27, 'max_depth': 6, 'n_estimators': 200}-8.128970 (6.796862) with: {'learning_rate': 0.27, 'max_depth': 7, 'n_estimators': 100}-8.138011 (6.797582) with: {'learning_rate': 0.27, 'max_depth': 7, 'n_estimators': 200}-8.291199 (6.426882) with: {'learning_rate': 0.3, 'max_depth': 4, 'n_estimators': 100}-8.336647 (6.448367) with: {'learning_rate': 0.3, 'max_depth': 4, 'n_estimators': 200}-8.148781 (6.531570) with: {'learning_rate': 0.3, 'max_depth': 5, 'n_estimators': 100}-8.181453 (6.528953) with: {'learning_rate': 0.3, 'max_depth': 5, 'n_estimators': 200}-8.488194 (6.555250) with: {'learning_rate': 0.3, 'max_depth': 6, 'n_estimators': 100}-8.499785 (6.552571) with: {'learning_rate': 0.3, 'max_depth': 6, 'n_estimators': 200}-8.432480 (6.699986) with: {'learning_rate': 0.3, 'max_depth': 7, 'n_estimators': 100}-8.437643 (6.698732) with: {'learning_rate': 0.3, 'max_depth': 7, 'n_estimators': 200}XGBR_GSCV_Shuffle_time： 137.92240889430929`

T2、调用XGBR_GSCV_Time()函数，调优+重要性可视化+特征选择模型

XGBR_model = XGBRegressor( learning_rate=0.2, max_depth= 2, n_estimators=100 )     #XGBR_GSCV_Time()函数，第一次得到最佳参数组合，输出准确度： 0.929254087319193

`Best: 0.8637 using {'learning_rate': 0.2, 'max_depth': 2, 'n_estimators': 100}XGBR_GSCV_Time score： 0.92951600308978-53.652829 (8.524886) with: {'learning_rate': 0.0001, 'max_depth': 2, 'n_estimators': 50}-53.111776 (8.432226) with: {'learning_rate': 0.0001, 'max_depth': 2, 'n_estimators': 100}-52.045728 (8.249767) with: {'learning_rate': 0.0001, 'max_depth': 2, 'n_estimators': 200}-49.976399 (7.896034) with: {'learning_rate': 0.0001, 'max_depth': 2, 'n_estimators': 400}-53.652829 (8.524886) with: {'learning_rate': 0.0001, 'max_depth': 4, 'n_estimators': 50}-53.111776 (8.432226) with: {'learning_rate': 0.0001, 'max_depth': 4, 'n_estimators': 100}-52.045728 (8.249767) with: {'learning_rate': 0.0001, 'max_depth': 4, 'n_estimators': 200}-49.976399 (7.896034) with: {'learning_rate': 0.0001, 'max_depth': 4, 'n_estimators': 400}-53.652829 (8.524886) with: {'learning_rate': 0.0001, 'max_depth': 6, 'n_estimators': 50}-53.111776 (8.432226) with: {'learning_rate': 0.0001, 'max_depth': 6, 'n_estimators': 100}-52.045728 (8.249767) with: {'learning_rate': 0.0001, 'max_depth': 6, 'n_estimators': 200}-49.976399 (7.896034) with: {'learning_rate': 0.0001, 'max_depth': 6, 'n_estimators': 400}-53.652829 (8.524886) with: {'learning_rate': 0.0001, 'max_depth': 8, 'n_estimators': 50}-53.111776 (8.432226) with: {'learning_rate': 0.0001, 'max_depth': 8, 'n_estimators': 100}-52.045728 (8.249767) with: {'learning_rate': 0.0001, 'max_depth': 8, 'n_estimators': 200}-49.976399 (7.896034) with: {'learning_rate': 0.0001, 'max_depth': 8, 'n_estimators': 400}-48.970063 (7.724229) with: {'learning_rate': 0.001, 'max_depth': 2, 'n_estimators': 50}-44.237571 (6.918364) with: {'learning_rate': 0.001, 'max_depth': 2, 'n_estimators': 100}-36.078697 (5.538471) with: {'learning_rate': 0.001, 'max_depth': 2, 'n_estimators': 200}-23.929587 (3.519751) with: {'learning_rate': 0.001, 'max_depth': 2, 'n_estimators': 400}-48.970063 (7.724229) with: {'learning_rate': 0.001, 'max_depth': 4, 'n_estimators': 50}-44.237571 (6.918364) with: {'learning_rate': 0.001, 'max_depth': 4, 'n_estimators': 100}-36.078697 (5.538471) with: {'learning_rate': 0.001, 'max_depth': 4, 'n_estimators': 200}-23.929587 (3.519751) with: {'learning_rate': 0.001, 'max_depth': 4, 'n_estimators': 400}-48.970063 (7.724229) with: {'learning_rate': 0.001, 'max_depth': 6, 'n_estimators': 50}-44.237571 (6.918364) with: {'learning_rate': 0.001, 'max_depth': 6, 'n_estimators': 100}-36.078697 (5.538471) with: {'learning_rate': 0.001, 'max_depth': 6, 'n_estimators': 200}-23.929587 (3.519751) with: {'learning_rate': 0.001, 'max_depth': 6, 'n_estimators': 400}-48.970063 (7.724229) with: {'learning_rate': 0.001, 'max_depth': 8, 'n_estimators': 50}-44.237571 (6.918364) with: {'learning_rate': 0.001, 'max_depth': 8, 'n_estimators': 100}-36.078697 (5.538471) with: {'learning_rate': 0.001, 'max_depth': 8, 'n_estimators': 200}-23.929587 (3.519751) with: {'learning_rate': 0.001, 'max_depth': 8, 'n_estimators': 400}-19.414644 (2.830758) with: {'learning_rate': 0.01, 'max_depth': 2, 'n_estimators': 50}-6.744672 (0.933997) with: {'learning_rate': 0.01, 'max_depth': 2, 'n_estimators': 100}-0.216053 (0.050337) with: {'learning_rate': 0.01, 'max_depth': 2, 'n_estimators': 200}0.848897 (0.024814) with: {'learning_rate': 0.01, 'max_depth': 2, 'n_estimators': 400}-19.414644 (2.830758) with: {'learning_rate': 0.01, 'max_depth': 4, 'n_estimators': 50}-6.743499 (0.932824) with: {'learning_rate': 0.01, 'max_depth': 4, 'n_estimators': 100}-0.254126 (0.091086) with: {'learning_rate': 0.01, 'max_depth': 4, 'n_estimators': 200}0.831512 (0.008093) with: {'learning_rate': 0.01, 'max_depth': 4, 'n_estimators': 400}-19.414644 (2.830758) with: {'learning_rate': 0.01, 'max_depth': 6, 'n_estimators': 50}-6.743499 (0.932824) with: {'learning_rate': 0.01, 'max_depth': 6, 'n_estimators': 100}-0.260028 (0.093910) with: {'learning_rate': 0.01, 'max_depth': 6, 'n_estimators': 200}0.829355 (0.015182) with: {'learning_rate': 0.01, 'max_depth': 6, 'n_estimators': 400}-19.414644 (2.830758) with: {'learning_rate': 0.01, 'max_depth': 8, 'n_estimators': 50}-6.743499 (0.932824) with: {'learning_rate': 0.01, 'max_depth': 8, 'n_estimators': 100}-0.258236 (0.092933) with: {'learning_rate': 0.01, 'max_depth': 8, 'n_estimators': 200}0.831777 (0.028036) with: {'learning_rate': 0.01, 'max_depth': 8, 'n_estimators': 400}0.852283 (0.003829) with: {'learning_rate': 0.1, 'max_depth': 2, 'n_estimators': 50}0.813154 (0.046960) with: {'learning_rate': 0.1, 'max_depth': 2, 'n_estimators': 100}0.829779 (0.037321) with: {'learning_rate': 0.1, 'max_depth': 2, 'n_estimators': 200}0.832717 (0.031505) with: {'learning_rate': 0.1, 'max_depth': 2, 'n_estimators': 400}0.785207 (0.061920) with: {'learning_rate': 0.1, 'max_depth': 4, 'n_estimators': 50}0.757671 (0.097880) with: {'learning_rate': 0.1, 'max_depth': 4, 'n_estimators': 100}0.772923 (0.083151) with: {'learning_rate': 0.1, 'max_depth': 4, 'n_estimators': 200}0.777985 (0.077499) with: {'learning_rate': 0.1, 'max_depth': 4, 'n_estimators': 400}0.800020 (0.031554) with: {'learning_rate': 0.1, 'max_depth': 6, 'n_estimators': 50}0.722744 (0.115322) with: {'learning_rate': 0.1, 'max_depth': 6, 'n_estimators': 100}0.718966 (0.120953) with: {'learning_rate': 0.1, 'max_depth': 6, 'n_estimators': 200}0.716761 (0.123083) with: {'learning_rate': 0.1, 'max_depth': 6, 'n_estimators': 400}0.816402 (0.004015) with: {'learning_rate': 0.1, 'max_depth': 8, 'n_estimators': 50}0.766141 (0.059941) with: {'learning_rate': 0.1, 'max_depth': 8, 'n_estimators': 100}0.756297 (0.069550) with: {'learning_rate': 0.1, 'max_depth': 8, 'n_estimators': 200}0.755626 (0.070178) with: {'learning_rate': 0.1, 'max_depth': 8, 'n_estimators': 400}0.855146 (0.003964) with: {'learning_rate': 0.2, 'max_depth': 2, 'n_estimators': 50}0.863665 (0.002430) with: {'learning_rate': 0.2, 'max_depth': 2, 'n_estimators': 100}0.862916 (0.000224) with: {'learning_rate': 0.2, 'max_depth': 2, 'n_estimators': 200}0.849430 (0.007344) with: {'learning_rate': 0.2, 'max_depth': 2, 'n_estimators': 400}0.758113 (0.097414) with: {'learning_rate': 0.2, 'max_depth': 4, 'n_estimators': 50}0.759158 (0.098429) with: {'learning_rate': 0.2, 'max_depth': 4, 'n_estimators': 100}0.754193 (0.102434) with: {'learning_rate': 0.2, 'max_depth': 4, 'n_estimators': 200}0.748421 (0.107894) with: {'learning_rate': 0.2, 'max_depth': 4, 'n_estimators': 400}0.780980 (0.061204) with: {'learning_rate': 0.2, 'max_depth': 6, 'n_estimators': 50}0.773959 (0.067553) with: {'learning_rate': 0.2, 'max_depth': 6, 'n_estimators': 100}0.773742 (0.067638) with: {'learning_rate': 0.2, 'max_depth': 6, 'n_estimators': 200}0.773425 (0.067856) with: {'learning_rate': 0.2, 'max_depth': 6, 'n_estimators': 400}0.804540 (0.032247) with: {'learning_rate': 0.2, 'max_depth': 8, 'n_estimators': 50}0.800325 (0.036309) with: {'learning_rate': 0.2, 'max_depth': 8, 'n_estimators': 100}0.800133 (0.036625) with: {'learning_rate': 0.2, 'max_depth': 8, 'n_estimators': 200}0.800134 (0.036625) with: {'learning_rate': 0.2, 'max_depth': 8, 'n_estimators': 400}0.804575 (0.055743) with: {'learning_rate': 0.3, 'max_depth': 2, 'n_estimators': 50}0.823723 (0.042951) with: {'learning_rate': 0.3, 'max_depth': 2, 'n_estimators': 100}0.832058 (0.027793) with: {'learning_rate': 0.3, 'max_depth': 2, 'n_estimators': 200}0.824320 (0.028952) with: {'learning_rate': 0.3, 'max_depth': 2, 'n_estimators': 400}0.684716 (0.174854) with: {'learning_rate': 0.3, 'max_depth': 4, 'n_estimators': 50}0.683423 (0.176741) with: {'learning_rate': 0.3, 'max_depth': 4, 'n_estimators': 100}0.676494 (0.183628) with: {'learning_rate': 0.3, 'max_depth': 4, 'n_estimators': 200}0.676418 (0.183173) with: {'learning_rate': 0.3, 'max_depth': 4, 'n_estimators': 400}0.533161 (0.294224) with: {'learning_rate': 0.3, 'max_depth': 6, 'n_estimators': 50}0.520398 (0.307576) with: {'learning_rate': 0.3, 'max_depth': 6, 'n_estimators': 100}0.520455 (0.307122) with: {'learning_rate': 0.3, 'max_depth': 6, 'n_estimators': 200}0.520411 (0.307169) with: {'learning_rate': 0.3, 'max_depth': 6, 'n_estimators': 400}0.666960 (0.156246) with: {'learning_rate': 0.3, 'max_depth': 8, 'n_estimators': 50}0.668800 (0.154254) with: {'learning_rate': 0.3, 'max_depth': 8, 'n_estimators': 100}0.668832 (0.154209) with: {'learning_rate': 0.3, 'max_depth': 8, 'n_estimators': 200}0.668832 (0.154209) with: {'learning_rate': 0.3, 'max_depth': 8, 'n_estimators': 400}XGBR_GSCV_Time_time： 61.41017997421118`