sklarn-learn 数据集


概述

我们将介绍 sklearn 中的数据集类, 模块包括用于加载数据集的实用程序, 包括假造和获取流行参数集的方法. 它还具有一些人工数据生成器.

sklearn.datasets

datasets.load_()

获取小规模数据集, 数据包含在 datasets 里.

datasets.fetch_()

获取大规模数据集, 需要从网上下载. 函数的第一个参数是 data_home, 表示数据集下载的目录, 默认是 ~/scikit_learn_data/, 要修改默认目录, 可以修改环境变量 SCIKIT_LEARN_DATA.

datasets.make_()

本地生成数据集.

返回类型

load 和 frtch 函数返回的数据类型是 datasets.base.Bruch, 本质上是一个 dict (字典). 它的键值对可通过对象的属性方式访问. 主要包含以下属性:


  • data: 特征数据数组, 是 [n_samples * n_futures] 的二维 numpy.ndarray 数组
  • target: 标签数组, 是 n_smaples 的一维 numpy.ndarray 数组
  • DESCR: 数据描述
  • feature_names: 特征名
  • target_names: 标签名

数据集目录

数据集目录可以通过​​datasets.get_data_home()​​​获取, ​​clear_data_home(data_home=None)​​删除所有下载数据.

返回 scikit 学习数据目录的路径. 这个文件夹被一些大的数据集转载器使用, 以避免下载数据. 默认情况下, 数据目录设置为用户主文件夹中名为 “scikit_learn_data” 的文件夹. 或者可以通过 “SCIKIT_LEARN_DATA” 环境变量或通过给出显示的文件夹路径以编程方式设置它. “~” 符号扩展到用户文件夹. 如果文件夹不存在, 则会自动创建.

获取小数据集

sklearn.datasets.load_iris

代码展示:

from sklearn.datasets import load_iris
# 实例化
li = load_iris()
# 调试输出
print("获取特征值: ", li.data)
print("目标值: ", li.target)
print("描述: " + li.DESCR)
输出结果:
获取特征值:  [[5.1 3.5 1.4 0.2]
 [4.7 3.2 1.3 0.2]
 [4.6 3.1 1.5 0.2]
 [5.4 3.9 1.7 0.4]
 [4.6 3.4 1.4 0.3]
 [4.4 2.9 1.4 0.2]
 [4.9 3.1 1.5 0.1]
 [5.4 3.7 1.5 0.2]
 [4.8 3.4 1.6 0.2]
 [5.7 4.4 1.5 0.4]
 [5.4 3.9 1.3 0.4]
 [5.1 3.5 1.4 0.3]
 [5.7 3.8 1.7 0.3]
 [5.1 3.8 1.5 0.3]
 [5.4 3.4 1.7 0.2]
 [5.1 3.7 1.5 0.4]
 [5.1 3.3 1.7 0.5]
 [4.8 3.4 1.9 0.2]
 [5.2 3.5 1.5 0.2]
 [5.2 3.4 1.4 0.2]
 [4.7 3.2 1.6 0.2]
 [4.8 3.1 1.6 0.2]
 [5.4 3.4 1.5 0.4]
 [5.2 4.1 1.5 0.1]
 [5.5 4.2 1.4 0.2]
 [4.9 3.1 1.5 0.2]
 [5.5 3.5 1.3 0.2]
 [4.9 3.6 1.4 0.1]
 [5.1 3.4 1.5 0.2]
 [4.5 2.3 1.3 0.3]
 [4.4 3.2 1.3 0.2]
 [5.1 3.8 1.9 0.4]
 [5.1 3.8 1.6 0.2]
 [4.6 3.2 1.4 0.2]
 [5.3 3.7 1.5 0.2]
 [6.4 3.2 4.5 1.5]
 [6.9 3.1 4.9 1.5]
 [6.5 2.8 4.6 1.5]
 [5.7 2.8 4.5 1.3]
 [6.3 3.3 4.7 1.6]
 [6.6 2.9 4.6 1.3]
 [5.2 2.7 3.9 1.4]
 [6.1 2.9 4.7 1.4]
 [5.6 2.9 3.6 1.3]
 [6.7 3.1 4.4 1.4]
 [6.2 2.2 4.5 1.5]
 [5.6 2.5 3.9 1.1]
 [5.9 3.2 4.8 1.8]
 [6.3 2.5 4.9 1.5]
 [6.1 2.8 4.7 1.2]
 [6.4 2.9 4.3 1.3]
 [6.8 2.8 4.8 1.4]
 [5.5 2.4 3.8 1.1]
 [5.8 2.7 3.9 1.2]
 [6.7 3.1 4.7 1.5]
 [6.3 2.3 4.4 1.3]
 [5.5 2.6 4.4 1.2]
 [5.6 2.7 4.2 1.3]
 [5.7 2.9 4.2 1.3]
 [6.2 2.9 4.3 1.3]
 [5.7 2.8 4.1 1.3]
 [5.8 2.7 5.1 1.9]
 [6.3 2.9 5.6 1.8]
 [4.9 2.5 4.5 1.7]
 [7.3 2.9 6.3 1.8]
 [6.7 2.5 5.8 1.8]
 [7.2 3.6 6.1 2.5]
 [6.4 2.7 5.3 1.9]
 [5.8 2.8 5.1 2.4]
 [6.4 3.2 5.3 2.3]
 [7.7 3.8 6.7 2.2]
 [7.7 2.6 6.9 2.3]
 [6.9 3.2 5.7 2.3]
 [6.3 2.7 4.9 1.8]
 [6.7 3.3 5.7 2.1]
 [6.2 2.8 4.8 1.8]
 [6.4 2.8 5.6 2.1]
 [7.4 2.8 6.1 1.9]
 [6.4 2.8 5.6 2.2]
 [6.3 2.8 5.1 1.5]
 [6.1 2.6 5.6 1.4]
 [6.3 3.4 5.6 2.4]
 [6.4 3.1 5.5 1.8]
 [6.9 3.1 5.4 2.1]
 [6.7 3.1 5.6 2.4]
 [6.9 3.1 5.1 2.3]
 [5.8 2.7 5.1 1.9]
 [6.8 3.2 5.9 2.3]
 [6.7 3.3 5.7 2.5]
 [6.2 3.4 5.4 2.3]
目标值:  [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2]
Iris plants dataset
--------------------
**Data Set Characteristics:**
    :Number of Instances: 150 (50 in each of three classes)
    :Number of Attributes: 4 numeric, predictive attributes and the class
    :Attribute Information:
        - sepal length in cm
        - sepal width in cm
        - petal length in cm
        - petal width in cm
        - class:
                - Iris-Setosa
                - Iris-Versicolour
                - Iris-Virginica

    :Summary Statistics:
    ============== ==== ==== ======= ===== ====================
                    Min  Max   Mean    SD   Class Correlation
    ============== ==== ==== ======= ===== ====================
    sepal length:   4.3  7.9   5.84   0.83    0.7826
    sepal width:    2.0  4.4   3.05   0.43   -0.4194
    petal length:   1.0  6.9   3.76   1.76    0.9490  (high!)
    petal width:    0.1  2.5   1.20   0.76    0.9565  (high!)
    ============== ==== ==== ======= ===== ====================
    :Missing Attribute Values: None
    :Class Distribution: 33.3% for each of 3 classes.
    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)
    :Date: July, 1988
Machine Learning Repository, which has two wrong data points.
This is perhaps the best known database to be found in the
data set contains 3 classes of 50 instances each, where each class refers to a
latter are NOT linearly separable from each other.
     Annual Eugenics, 7, Part II, 179-188 (1936); also in "Contributions to
     Mathematical Statistics" (John Wiley, NY, 1950).
     Structure and Classification Rule for Recognition in Partially Exposed
     on Information Theory, May 1972, 431-433.
     conceptual clustering system finds 3 classes in the data.
   - Many, many more ...

数据集进行分割

sklearn.model_selection.train_test_split(*arrays,**options)

  • x 数据集的特征值
  • y 数据集的目标值
  • test_size 测试集的大小, 一般为 float
  • random_state 随机数种子, 不同的种子会造成不同的随机采样结果. 相同的种子采样结果相同
  • return 训练集特征值, 测试集特征值. 训练标签, 测试标签 (默认随机取)

获取大数据集

sklearn.datasets.fetch_20newsgroups(data_home = None, subset = "train")

subset: “train” 或者 “test”, “all” 可选, 选择要加载的数据集. 训练集的 “训练”, 测试集的 “测试”, 两者的 “全部”.

代码展示:

from sklearn.datasets import fetch_20newsgroups
# 实例化
news = fetch_20newsgroups(subset="all")
# 调试输出
print(news.data)
print(news.target)
输出结果:

回归数据集:

from sklearn.datasets import load_boston
# 实例化
lb = load_boston()
# 调试输出
print("获取特征值: ", lb.data)
print("获取目标值: ", lb.target)
print("描述: " + lb.DESCR)
输出结果:
 ...
 18.2 19.9 23.1 17.5 20.2 18.2 13.6 19.6 15.2 14.5 15.6 13.9 16.6 14.8
 43.8 33.2 27.5 26.5 18.6 19.3 20.1 19.5 19.5 20.4 19.8 19.4 21.7 22.8
 23.8 22.3 17.4 19.1 23.1 23.6 22.6 29.4 23.2 24.6 29.9 37.2 39.8 36.2
 13.1 10.2 10.4 10.9 11.3 12.3  8.8  7.2 10.5  7.4 10.2 11.5 15.1 23.2
 11.7 13.4  9.6  8.7  8.4 12.8 10.5 17.1 18.4 15.4 10.8 11.8 14.9 12.6
  8.1 13.6 20.1 21.8 24.5 23.1 19.7 18.3 21.2 17.5 16.8 22.4 20.6 23.9
Boston house prices dataset
---------------------------
**Data Set Characteristics:**  
    :Number of Instances: 506 
    :Attribute Information (in order):
        - CRIM     per capita crime rate by town
        - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.
        - INDUS    proportion of non-retail business acres per town
        - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
        - NOX      nitric oxides concentration (parts per 10 million)
        - RM       average number of rooms per dwelling
        - AGE      proportion of owner-occupied units built prior to 1940
        - DIS      weighted distances to five Boston employment centres
        - RAD      index of accessibility to radial highways
        - TAX      full-value property-tax rate per $10,000
        - PTRATIO  pupil-teacher ratio by town
        - B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town
        - LSTAT    % lower status of the population
        - MEDV     Median value of owner-occupied homes in $1000's
    :Missing Attribute Values: None
This is a copy of UCI ML housing dataset.
https://archive.ics.uci.edu/ml/machine-learning-databases/housing/
This dataset was taken from the StatLib library which is maintained at Carnegie Mellon University.
pages 244-261 of the latter.
The Boston house-price data has been used in many machine learning papers that address regression