tensorflow 安装 单次国内源

教是学最好的方式，fighting

tensorflow V2.2的安装

• tensorflow常见的安装途径包括两个：pip安装和conda安装（强烈推荐后者，后者具备如下安装优势：1. 速度快，2. 自动安装与tensorflow版本配套的cuda和cudnn【手动安装的时候一定要参考以下链接，找到对应的版本https:///install/source#linux】）
• conda安装过程

1. 使用conda创建虚拟环境变量  conda create -n xx   （这里一定不要指定python版本，以防安装tensorflow时出现包冲突）
2. 安装tensorflow-gpu    conda install tensorflow-gpu==2.2.0  (这里可以先通过 conda list tensorflow-gpu 查看是否安装过tensorflow-gpu， 然后使用conda search tensorflow-gpu 查看可以安装的tensorflow的版本）

基于conda安装的tensorflow 2.2 在运行过程中啪啪打脸，故又采用了pip安装tensorflow，同时手动安装cuda和cudnn。pip安装过程中要注意的问题：

1. 查看tensorflow-gpu各版本对应的cuda和cudnn

2. 若运行过程中提示以下内容：tensorflow-gpu: Could not load dynamic library '.10.1'，请根据https:///tensorflow/tensorflow/issues/39132#issuecomment-650565521

进行修改。

tensorflow V2.2的核心概念

tf.modules，layers and models

In TensorFlow, most high-level implementations of layers and models, such as Keras or Sonnet, are built on the same foundational class: tf.Module.

Tensorflow的高级接口（layers 和 models）都建立在tf.module的基础上，tf.module会维护对象内部所有的变量。keras.Layer和keras.Model都是tf.Module的子类。

tf.function

https:///guide/function

In TensorFlow 2, eager execution is turned on by default. The user interface is intuitive and flexible (running one-off operations is much easier and faster), but this can come at the expense of performance and deployability.

You can use tf.function to make graphs out of your programs. It is a transformation tool that creates Python-independent dataflow graphs out of your Python code. This will help you create performant and portable models, and it is required to use SavedModel.

This guide will help you conceptualize how tf.function works under the hood so you can use it effectively.

The main takeaways and recommendations are:

• Debug in eager mode, then decorate with @tf.function.
• Don't rely on Python side effects like object mutation or list appends.
• tf.function works best with TensorFlow ops; NumPy and Python calls are converted to constants.

eager式的顺序执行保证了代码的灵活性，但是代码的性能和部署的便利性丧失。 tf.function相当于一个转化工具，他会将其block内的代码生成一个不依赖python的可独立运行的图。（解释：这个现象同时存在于V1.14里，模型第一次运行时间如此之长就是因为要生成这张可以独立运行的图。）

1. Function的运行速度

Functions can be faster than eager code, especially for graphs with many small ops. But for graphs with a few expensive ops (like convolutions), you may not see much speedup.

从下方的实验结果来看，使用@tf.function之后的运行时间反而更长，这意味着模型采用eager excution的方式对应的推理时间可能会更短（需要进一步验证）

import timeit
conv_layer = tf.keras.layers.Conv2D(100, 3)

@tf.function
def conv_fn(image):
  return conv_layer(image)

image = tf.zeros([1, 200, 200, 100])
# warm up
conv_layer(image); conv_fn(image)
print("Eager conv:", timeit.timeit(lambda: conv_layer(image), number=10))
print("Function conv:", timeit.timeit(lambda: conv_fn(image), number=10))
print("Note how there's not much difference in performance for convolutions")


Eager conv: 0.002407395000091128
Function conv: 0.004000883000117028
Note how there's not much difference in performance for convolutions

2. Function的tracking机制

• Note that if you repeatedly call a Function with the same argument type, TensorFlow will reuse a previously traced graph, as the generated graph would be identical.

当function的输入的数据类型确定时，function对应的graph会被重用。（和1.14的原理一样，但是为什么session.run的输入的内容变化，数据类型不变，图也会重新生成呢？2.2是否也具有这样的特点需要进一步测量）

每个被tracking的function都会生成一个concrete function，concrete function对应一个graph，我们可以通过graph的若干属性得到图相关的内容。

@tf.function()
def fun_string(a):
    return a+a
fun = fun_string.get_concrete_function(tf.constant("a"))
for node in fun.graph.as_graph_def().node:
  print(f'{node.input} -> {}')


输出：
[] -> a
['a', 'a'] -> add
['add'] -> Identity

3. Function的reuse机制

https:///guide/function#tracing_semantics

A Function determines whether to reuse a traced concrete function by computing a cache key from an input's args and kwargs.

• The key generated for a tf.Tensor argument is its shape and dtype.
• Starting in TensorFlow 2.3, the key generated for a tf.Variable argument is its id().
• The key generated for a Python primitive is its value. The key generated for nested dicts, lists, tuples, namedtuples, and attrs is the flattened tuple. (As a result of this flattening, calling a concrete function with a different nesting structure than the one used during tracing will result in a TypeError).
• For all other Python types, the keys are based on the object id() so that methods are traced independently for each instance of a class.

function底层的graph在以下情况下会被重用：

•  function输入的数据类型为tf.Tensor，且tensor的shape和type相同
• function输入的数据（实参）为python数据

1. 实参为python的基本数据类型时，形参的值没有发生变化
2. 形参为python的dict，list，tuple或string array，flattened后的内容不发生变化（元素的值和位置都没有发生变化）
3. 形参为python的其他对象（object）， object的id（）没发生变化（The identity of an object is an integer, which is guaranteed to be unique and constant for this object during its lifetime）

要尽可能的让graph重用以提高程序的执行效率

@tf.function(input_signature=(tf.TensorSpec(shape=[None], dtype=tf.int32),)) def g(x):   print('Tracing with', x)   return x # No retrace! print(g(tf.constant([1, 2, 3]))) print(g(tf.constant([1, 2, 3, 4, 5])))

• Cast Python arguments to Tensors to reduce retracing.

Often, Python arguments are used to control hyperparameters and graph constructions - for example, num_layers=10 or training=True or nonlinearity='relu'. So if the   Python argument changes, it makes sense that you'd have to retrace the graph.

def train_one_step(): pass @tf.function def train(num_steps): print("Tracing with num_steps = ", num_steps) tf.print("Executing with num_steps = ", num_steps) for _ in tf.range(num_steps): train_one_step() print("Retracing occurs for different Python arguments.") train(num_steps=10) #重新生成graph train(num_steps=20) print() print("Traces are reused for Tensor arguments.") train(num_steps=tf.constant(10)) train(num_steps=tf.constant(20)) # 输出结果 Retracing occurs for different Python arguments. Tracing with num_steps = 10 Executing with num_steps = 10 Tracing with num_steps = 20 Executing with num_steps = 20 Traces are reused for Tensor arguments. Tracing with num_steps = Tensor("num_steps:0", shape=(), dtype=int32) Executing with num_steps = 10 Executing with num_steps = 20 #分析结果 ''' function有控制机制，同一个function只会输入一次print的内容。 第二组中print()只输入了一次，tf.print()输出了多次。 由此可以看出只生成了一个graph。 '''

加强function reuse的方法：

• 实参又python类行的参数转化成Tensor类行的参数
• 将funtion的形参指定如下形式（这要求实参必须为tf集成的数据类型）

tf.TensorSpec(shape=[None], dtype=tf.int32),)

4. 加强function reuse的方法

https:///guide/function#controlling_retracing

Retracing helps ensures that TensorFlow generates correct graphs for each set of inputs. However, tracing is an expensive operation! If your Function retraces a new graph for every call, you'll find that your code executes more slowly than if you didn't use tf.function.

To control the tracing behavior, you can use the following techniques:

• Specify input_signature in tf.function to limit tracing.
• Specify a [None] dimension in tf.TensorSpec to allow for flexibility in trace reuse.

Since TensorFlow matches tensors based on their shape, using a None dimension as a wildcard will allow Functions to reuse traces for variably-sized input. Variably-sized input can occur if you have sequences of different length, or images of different sizes for each batch (See Transformer and Deep Dream tutorials for example).

调用特定完整模型

https://keras.io/api/applications/#available-models

调用特定不完整模型

vgg19 = tf.keras.applications.VGG19()
features_list = [layer.output for layer in vgg19.layers]
feat_extraction_model = keras.Model(inputs=vgg19.input, outputs=features_list)
img = np.random.random((1, 224, 224, 3)).astype("float32")
extracted_features = feat_extraction_model(img)

keras model和tf.function的关系

keras.Model是keras集成的用于model管理的高级接口，tf.function下调用某个model相当于为这个model创建graph。但是，一定要注意model内部variable的使用。

#================== 继承tf.keras.Model，创建子类===============
# Create an oveerride model to classify pictures
class SequentialModel(tf.keras.Model):
  def __init__(self, **kwargs):
    super(SequentialModel, self).__init__(**kwargs)
    self.flatten = tf.keras.layers.Flatten(input_shape=(28, 28))
    self.dense_1 = tf.keras.layers.Dense(128, activation="relu")
    self.dropout = tf.keras.layers.Dropout(0.2)
    self.dense_2 = tf.keras.layers.Dense(10)

  def call(self, x):
    x = self.flatten(x)
    x = self.dense_1(x)
    x = self.dropout(x)
    x = self.dense_2(x)
    return x

input_data = tf.random.uniform([60, 28, 28])

eager_model = SequentialModel()
graph_model = tf.function(eager_model)

print("Eager time:", timeit.timeit(lambda: eager_model(input_data), number=10000))
print("Graph time:", timeit.timeit(lambda: graph_model(input_data), number=10000))

#运行结果

Eager time: 5.330957799000316
Graph time: 3.0375442899999143


#==================调用keras内的api=====================
input = tf.keras.layers.Input(shape=(28, 28))
x = tf.keras.layers.Flatten(input_shape=(28, 28))(input)
x = tf.keras.layers.Dense(128, activation="relu")(x)
x = tf.keras.layers.Dropout(0.2)(x)
x = tf.keras.layers.Dense(10)(x)
y = tf.keras.Model(input,x,name="test")
input_data = tf.random.uniform([60, 28, 28])
graph_model = tf.function(y)
print("Eager time:", timeit.timeit(lambda: y(input_data), number=10000))
print("Graph time:", timeit.timeit(lambda: graph_model(input_data), number=10000))
# 运行结果
Eager time: 11.091867479000939
Graph time: 3.8052064549992792

V2.2和V1.14的若干区别

https:///guide/effective_tf2#use_tfconfigexperimental_run_functions_eagerly_when_debugging

eager execution

1. V1.14需要调用session.run运行图，V2.2 直接调用tf.function即可，且function会自动优化图。

# TensorFlow 1.X
outputs = session.run(f(placeholder), feed_dict={placeholder: input})
# TensorFlow 2.0
outputs = f(input)

1. V1.14不需要user维护其创建的tf.Vairable，tensorflow会持续track所有variable；V2.2 将所有tf.Variable视为object，需要用户自己维护，如果variable失去指针，该变量就会被回收。V2.2为了方便变量管理引入了keras layer和model https:///guide/effective_tf2#use_keras_layers_and_models_to_manage_variables