3080 装 cuda 的坑

1 你要用 TensorFlow2.4 的 gpu 版本

2 cuda10.1 3080不认,会出现不报错也不训练的情况

3 cuda要用 11.0 ,cudnn是 8.0.5

4 numpy报错的话

pip install numpy==1.19.3

5 然后cudnn目前还无法通过 conda、pip之类的安装,下载之后copy文件

lib文件

G:\迅雷下载\cudnn-11.0-windows-x64-v8.0.5.39\cuda\lib\x64

D:\ProgramWorks\Miniconda3\envs\tf24\Library\lib

include文件

G:\迅雷下载\cudnn-11.0-windows-x64-v8.0.5.39\cuda\include

D:\ProgramWorks\Miniconda3\envs\tf24\Library\include

bin文件

G:\迅雷下载\cudnn-11.0-windows-x64-v8.0.5.39\cuda\bin

D:\ProgramWorks\Miniconda3\envs\tf24\Library\bin

import tensorflow as tf
import os

os.environ[‘TF_CPP_MIN_LOG_LEVEL’] = ‘2’  # 不显示等级2以下的提示信息

print(‘GPU’, tf.test.is_gpu_available())

a = tf.constant(2.0)
b = tf.constant(4.0)
print(a + b)

pip install tensorflow-gpu==2.4 –upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple

pip install tensorflow-gpu==2.4 –upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple some-package

 conda install cudatoolkit=11

然后就是下载 cudnn,拷贝

pip install sklearn
pip install matplotlib
pip install opencv-python
pip install numpy==1.19.3
jupyter notebook
pip install pandas

numpy reshape

numpy的reshape方法平时用的地方很多,但是有好多人实际上并不了解这个方法到底在做什么

1. reshape不会修改数组中的任何数据

2. shape的形状有时候会比较复杂,比如 (2,3,4,5,6)

这种看起来比较复杂,其实理解起来也很容易,可以把它想象成俄罗斯套娃,

第一层有2个元素

这两个元素中每个又包含3个元素

这三个元素中每个又包含4个元素

这样一层又一层包了5层

如何训练你自己的yolov3标注数据

这里我使用的是猫狗图片集

1 安装vott

https://github.com/Microsoft/VoTT/releases

2 创建vott项目

3 导出格式设置为 csv

4 打开自动检测,这样可以省下不少手动标注工作

这里只是一个演示,如果是自己的项目,这里还是需要手动做标注的

5 克隆

git clone https://github.com/AntonMu/TrainYourOwnYOLO.git

然后别忘了安装一下依赖包 pip install -r requirements.txt

6 拷贝到处的 图片文件夹到

TrainYourOwnYOLO/Data/Source_Images/Training_Images

这里需要拷贝的是 vott-csv-export ,这个文件夹的根路径下应该包含

dogcat-export.csv 和 若干 jpeg 图片

7 对图片进行处理

把vott生成的 vott-csv-export 文件夹拷贝到 Training_Images 下,然后调用

python Convert_to_YOLO_format.py

会得到 Data/Model_Weights/data_classes.txt文件

8 下载默认权重

cd 到 TrainYourOwnYOLO/2_Training

python Download_and_Convert_YOLO_weights.py

脚本启动之后会在/2_Training/src/keras_yolo3 路径下 下载 yolov3.weights

下载完成之后会拷贝成 yolo.h5

9 然后就可以使用 文件夹2 中的

python Train_YOLO.py 

来进行训练了

( 这里由于我之前的文件路径带了 – 减号,会发生找不到文件 )

把 减号 去掉之后,重新依次调用

python Convert_to_YOLO_format.py

python Download_and_Convert_YOLO_weights.py ( 其实不用 )

python Train_YOLO.py 

最终会生成一个 trained_weights_final.h5

然后就可以使用

python Detector.py

进行预测了

一共2000张图片,使用

RTX6000 24GB 显存的话,100秒每 epoch

CPU 就慢多了, 540秒每 epoch 

要使用 gpu的话

conda install cudatoolkit=10.1

conda install cudnn  ( 当前是 默认 7.6 )

Rasa forms

一种常见情形是,使用聊天机器人来搜集用户提供的信息,在rasa中,一般这种行为叫做

slot filling

需要在 policy 中添加

policies:

  - name: FormPolicy

  - name: MemoizationPolicy

  - name: TEDPolicy

    max_history: 5

    epochs: 100

  - name: MappingPolicy

这里我使用的是官方例子,不过做了一些小修改 

## request restaurant happy path

* request_restaurant

    - restaurant_form

    - form{"name": "restaurant_form"}

    - form{"name": null}

Docker GPU

sudo apt install docker.io

sudo usermod -a -G docker lizhe

然后正常退出,重启docker保证能正常使用

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -

curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

sudo apt-get update

sudo apt-get install -y nvidia-container-toolkit

sudo systemctl restart docker

docker run –gpus all nvidia/cuda:10.0-base nvidia-smi

docker run –gpus all –name tensorflow1 -it -p 8080:8888 tensorflow/tensorflow:latest-gpu-py3-jupyter

import tensorflow as tf
import os

os.environ[‘TF_CPP_MIN_LOG_LEVEL’]=’2′

a = tf.constant(1.)
b = tf.constant(2.)
print(a+b)

print(‘GPU:’, tf.test.is_gpu_available())

Rasa 3 NLU

The first piece of a Rasa assistant is an NLU model. NLU stands for Natural Language Understanding, which means turning user messages into structured data. To do this with Rasa, you provide training examples that show how Rasa should understand user messages, and then train a model by showing it those examples.

Run the code cell below to see the NLU training data created by the rasa init command:

1

cat data/nlu.md

lizhe@ubuntu:~/rasa_workspace$ cat data/nlu.md

## intent:greet

– hey

– hello

– hi

– good morning

– good evening

– hey there

## intent:goodbye

– bye

– goodbye

– see you around

– see you later

## intent:affirm

– yes

– indeed

– of course

– that sounds good

– correct

## intent:deny

– no

– never

– I don’t think so

– don’t like that

– no way

– not really

## intent:mood_great

– perfect

– very good

– great

– amazing

– wonderful

– I am feeling very good

– I am great

– I’m good

## intent:mood_unhappy

– sad

– very sad

– unhappy

– bad

– very bad

– awful

– terrible

– not very good

– extremely sad

– so sad

## intent:bot_challenge

– are you a bot?

– are you a human?

– am I talking to a bot?

– am I talking to a human?

lizhe@ubuntu:~/rasa_workspace$ 


Rasa NLU: Language Understanding for Chatbots and AI assistants

Rasa NLU is an open-source natural language processing tool for intent classification, response retrieval and entity extraction in chatbots. For example, taking a sentence like

“I am looking for a Mexican restaurant in the center of town”

and returning structured data like

{

  “intent”: “search_restaurant”,

  “entities”: {

    “cuisine” : “Mexican”,

    “location” : “center”

  }

}


Using NLU Only

If you want to use Rasa only as an NLU component, you can!

Training NLU-only models

To train an NLU model only, run:

rasa train nlu

This will look for NLU training data files in the data/ directory and saves a trained model in the models/ directory. The name of the model will start with nlu-.

To try out your NLU model on the command line, use the rasa shell nlu command:

rasa shell nlu

This will start the rasa shell and ask you to type in a message to test. You can keep typing in as many messages as you like.

Alternatively, you can leave out the nlu argument and pass in an nlu-only model directly:

rasa shell -m models/nlu-20190515-144445.tar.gz

Running an NLU server

To start a server with your NLU model, pass in the model name at runtime:

rasa run –enable-api -m models/nlu-20190515-144445.tar.gz

You can then request predictions from your model using the /model/parse endpoint. To do this, run:

curl localhost:5005/model/parse -d ‘{“text”:”hello”}’

Rasa 2 New project

The first step is to create a new Rasa project. To do this, run:

rasa init --no-prompt

The rasa init command creates all the files that a Rasa project needs and trains a simple bot on some sample data. If you leave out the –no-prompt flag you will be asked some questions about how you want your project to be set up.

This creates the following files:

__init__.pyan empty file that helps python find your actions
actions.pycode for your custom actions
config.yml ‘*’configuration of your NLU and Core models
credentials.ymldetails for connecting to other services
data/nlu.md ‘*’your NLU training data
data/stories.md ‘*’your stories
domain.yml ‘*’your assistant’s domain
endpoints.ymldetails for connecting to channels like fb messenger
models/<timestamp>.tar.gzyour initial model

The most important files are marked with a ‘*’. You will learn about all of these in this tutorial.

Rasa 1 install

Quick Installation

pip3 install rasa 

1. Create a New Project

rasa init --no-prompt 

2. View Your NLU Training Data

cat data/nlu.md 

3. Define Your Model Configuration

cat config.yml 

4. Write Your First Stories

## story1

* greet

   - utter_greet

5. Define a Domain

cat domain.yml 

6. Train a Model

 

rasa train

echo "Finished training."

7. Test Your Assistant

 

rasa test

echo "Finished running tests."

8. Talk to Your Assistant

rasa shell 

Tensorflow 对数几率回归

线性回归模型所预测的是一个连续值或任意实数

对数几率回归则用于预测 Y/N 

f(x) = 1/(1+e-x)

随着x值(横轴)的变化, y值(纵轴)会在0~1 这个区间内变化, 这里1表示Y, 0表示位N 

这个函数接收单个输入值所以这里如果我们需要使用多个输入值的话,需要将多个值合并成单个值使用

例如如果数据中是 

姓名    年龄    性别

Tom    20       Male

可能需要合并成 Tom_20_Male之后才能放入上面的公式参与计算

这里的例子是计算泰坦尼克的乘客的幸存概率,原始数据为

PassengerId,Survived,Pclass,Name,Sex,Age,SibSp,Parch,Ticket,Fare,Cabin,Embarked
1,0,3,”Braund, Mr. Owen Harris”,male,22,1,0,A/5 21171,7.25,,S
2,1,1,”Cumings, Mrs. John Bradley (Florence Briggs Thayer)”,female,38,1,0,PC 17599,71.2833,C85,C
3,1,3,”Heikkinen, Miss. Laina”,female,26,0,0,STON/O2. 3101282,7.925,,S

下载地址 https://www.kaggle.com/c/titanic/data (需要注册)

你也可以直接从本网站获取 http://lizhe.name/sites/default/files/download/train.csv

其中我们只抽取三种数据作为计算依据

仓位等级 性别 和 年龄

这些数据均为 “属性数据” 即没有关联性,只是表示 是 或者 否 的数据

所以最好的办法是将他们作为一个N维的布尔值来使用然后得到 5 个布尔值

is_first_class  是否为一等舱

is_second_class 是否为二等舱

is_third_class 是否为三等舱

gender 性别 (0为男性,1为女性)

age 年龄

对于上面说到的第一步我们需要将5个属性值合并成一个值

W = tf.Variable(tf.zeros([5, 1]), name="weights")#变量权值

b = tf.Variable(0., name="bias")#线性函数常量,模型偏置

def combine_inputs(X):#输入值合并

    print "function: combine_inputs"

    return tf.matmul(X, W) + b

这里因为每行有5个数据(列), 所以需要一个使用一个[5,1]的向量作为乘数 (5行1列)

然后加上偏移量b

我们得到一个 N行5列的矩阵 和 一个 5行1列的矩阵的乘积 与 偏移量b 的和

然后将合并函数导入推断函数

def inference(X):#计算返回推断模型输出(数据X)

    print "function: inference"

    return tf.sigmoid(combine_inputs(X))#调用概率分布函数

计算损失的函数为 交叉熵损失函数

ef loss(X, Y):#计算损失(训练数据X及期望输出Y)

    print "function: loss"

    #求平均值
    return tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(combine_inputs(X),Y))

接着定义读取csv文件的函数

def read_csv(batch_size, file_name, record_defaults):#从csv文件读取数据,加载解析,创建批次读取张量多行数据

    filename_queue = tf.train.string_input_producer([os.path.join(os.getcwd(), file_name)])

    reader = tf.TextLineReader(skip_header_lines=1)

    key, value = reader.read(filename_queue)

    decoded = tf.decode_csv(value, record_defaults=record_defaults)#字符串(文本行)转换到指定默认值张量列元组,为每列设置数据类型

    return tf.train.shuffle_batch(decoded, batch_size=batch_size, capacity=batch_size * 50, min_after_dequeue=batch_size)#读取文件,加载张量batch_size行

key, value = reader.read(filename_queue) 会读取csv文件中的一行, 内容保存到value变量中

此时 print(key,”::”,value) 会得到 

(<tf.Tensor ‘ReaderRead:0’ shape=() dtype=string>, ‘::’, <tf.Tensor ‘ReaderRead:1’ shape=() dtype=string>)

也就是一行数据

这里你可能会奇怪这个read方法命名没有在loop循环中, 是如何被调用了多次的

tensorflow的执行和其他框架有一些不一样, 它的代码并不是实时调用的逻辑代码, 而是为了生成数据流图而设计的

这里只是定义了一个图(流程)

train_op = train(total_loss)  

                                                  ==> total_loss = loss(X, Y)

                                                                                                   ==> X, Y = inputs()

                                                                                                                                      ==> read_csv(…)

真正的运算行为要等到sess.run([train_op]) 执行才会完整的开始调用这些逻辑, 而sess.run([train_op])这条语句恰恰就在loop循环中

decoded = tf.decode_csv(value, record_defaults=record_defaults) 把上一条语句得到的内容value转换成tensor对象的列
print(decoded) 输出得到一共12个tensor对象,分别对应不同的数据类型

[<tf.Tensor 'DecodeCSV:0' shape=() dtype=float32>,

<tf.Tensor 'DecodeCSV:1' shape=() dtype=float32>,

<tf.Tensor 'DecodeCSV:2' shape=() dtype=int32>,

<tf.Tensor 'DecodeCSV:3' shape=() dtype=string>,

<tf.Tensor 'DecodeCSV:4' shape=() dtype=string>,

<tf.Tensor 'DecodeCSV:5' shape=() dtype=float32>,

<tf.Tensor 'DecodeCSV:6' shape=() dtype=float32>,

<tf.Tensor 'DecodeCSV:7' shape=() dtype=float32>,

<tf.Tensor 'DecodeCSV:8' shape=() dtype=string>,

<tf.Tensor 'DecodeCSV:9' shape=() dtype=float32>,

<tf.Tensor 'DecodeCSV:10' shape=() dtype=string>,

<tf.Tensor 'DecodeCSV:11' shape=() dtype=string>]

return tf.train.shuffle_batch(decoded, batch_size=batch_size, capacity=batch_size * 50, min_after_dequeue=batch_size)

Creates batches by randomly shuffling tensors.

也就是对原数据进行”洗牌”, 有一个需要特别注意的地方reader.read函数本身返回一行数据

但是这里因为后面调用的时候传入的batch_size参数为100,所以在洗牌动作运行结束之后,我们已经读取了100行数据

(当然这些代码也要等到真正run的时候才开始执行)

inputs() 函数会从read_csv函数的返回值中获得数据并赋值给变量

需要注意的是这里得到了12个tensor对象, 每个tensor对象都包含了100个数据 (可以理解成向量)

tensor对象_1  (passenger_id)id_1, id_2, id_3 … id_100
tensor对象_2  (survived)survived_1, survived_2 … survived_100
tensor对象_3  (passenger_id)pclass_1, pclass_2 … pclass_100

passenger_id, survived, pclass, name, sex, age, sibsp, parch, ticket, fare, cabin, embarked = \
        read_csv(100, “train.csv”, [[0.0], [0.0], [0], [“”], [“”], [0.0], [0.0], [0.0], [“”], [0.0], [“”], [“”]])

然后将除了年龄之外的数据(字符串) 转换成数字, 年龄本身就是数字所以不用转换了

这些转换并不是转行单一常量,而是在转换100行1列的矩阵

    is_first_class = tf.to_float(tf.equal(pclass, [1]))#一等票

    is_second_class = tf.to_float(tf.equal(pclass, [2]))#二等票

    is_third_class = tf.to_float(tf.equal(pclass, [3]))#三等票

    gender = tf.to_float(tf.equal(sex, ["female"]))#性别,男性为0,女性为1

然后是tf.pack函数

tf.pack([is_first_class,is_second_class,is_third_class,gender,age])

tf.pack(values, axis=0, name=”pack”) 
Packs a list of rank-R tensors into one rank-(R+1) tensor 
将一个R维张量列表沿着axis轴组合成一个R+1维的张量。

  # 'x' is [1, 4]
  # 'y' is [2, 5]
  # 'z' is [3, 6]
  pack([x, y, z]) => [[1, 4], [2, 5], [3, 6]]  # Pack along first dim.
  pack([x, y, z], axis=1) => [[1, 2, 3], [4, 5, 6]]

pack函数的输出为

Tensor(“pack:0”, shape=(5, 100), dtype=float32)

这个函数将输入的 5个tensor对象组 合并成为 一个5行的矩阵, 因为每行有一个tensor对象,而每个tensor对象包含100个值,所以看起来是这样的

[

    [tensor_one_1 …. 100],

    [tensor_two_1 …. 100],

    [tensor_three_1 …. 100],

    …

]

然后对5行100列的矩阵进行转置得到一个100行5列的矩阵

x =  [[1 2 3]
        [4 5 6]]


tf.transpose(x) ==> [[1 4]
                                [2 5]
                                [3 6]]

因为属性为100行5列的矩阵

所以结论也需要转换成对应的格式 (100行1列)

survived = tf.reshape(survived, [100, 1])

# -*- coding: utf-8 -*-
import tensorflow as tf

import os

#参数变量初始化

W = tf.Variable(tf.zeros([5, 1]), name="weights")#变量权值

b = tf.Variable(0., name="bias")#线性函数常量,模型偏置

def combine_inputs(X):#输入值合并

    print "function: combine_inputs"

    return tf.matmul(X, W) + b

def inference(X):#计算返回推断模型输出(数据X)

    print "function: inference"

    return tf.sigmoid(combine_inputs(X))#调用概率分布函数

def loss(X, Y):#计算损失(训练数据X及期望输出Y)

    print "function: loss"

    #求平均值
    return tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(combine_inputs(X),Y))

def read_csv(batch_size, file_name, record_defaults):#从csv文件读取数据,加载解析,创建批次读取张量多行数据

    filename_queue = tf.train.string_input_producer([os.path.join(os.getcwd(), file_name)])

    reader = tf.TextLineReader(skip_header_lines=1)

    key, value = reader.read(filename_queue)

    decoded = tf.decode_csv(value, record_defaults=record_defaults)#字符串(文本行)转换到指定默认值张量列元组,为每列设置数据类型

    return tf.train.shuffle_batch(decoded, batch_size=batch_size, capacity=batch_size * 50, min_after_dequeue=batch_size)#读取文件,加载张量batch_size行

def inputs():#读取或生成训练数据X及期望输出Y

    print "function: inputs"

    #数据来源:https://www.kaggle.com/c/titanic/data

    #模型依据乘客年龄、性别、船票等级推断是否能够幸存

    passenger_id, survived, pclass, name, sex, age, sibsp, parch, ticket, fare, cabin, embarked = \
        read_csv(100, "train.csv", [[0.0], [0.0], [0], [""], [""], [0.0], [0.0], [0.0], [""], [0.0], [""], [""]])

    #转换属性数据

    is_first_class = tf.to_float(tf.equal(pclass, [1]))#一等票

    is_second_class = tf.to_float(tf.equal(pclass, [2]))#二等票

    is_third_class = tf.to_float(tf.equal(pclass, [3]))#三等票

    gender = tf.to_float(tf.equal(sex, ["female"]))#性别,男性为0,女性为1

    #所有特征排列矩阵,矩阵转置,每行一样本,每列一特征
    features = tf.transpose(tf.pack([is_first_class,is_second_class,is_third_class,gender,age]))

    survived = tf.reshape(survived, [100, 1])

    return features, survived

def train(total_loss):#训练或调整模型参数(计算总损失)

    print "function: train"

    learning_rate = 0.01

    return tf.train.GradientDescentOptimizer(learning_rate).minimize(total_loss)

def evaluate(sess, X, Y):#评估训练模型

    print "function: evaluate"

    predicted = tf.cast(inference(X) > 0.5, tf.float32)#样本输出大于0.5转换为正回答

    print sess.run(tf.reduce_mean(tf.cast(tf.equal(predicted, Y), tf.float32)))#统计所有正确预测样本数,除以批次样本总数,得到正确预测百分比

#会话对象启动数据流图,搭建流程

sess = tf.Session()

print "Session: start"

init = tf.initialize_all_variables()
sess.run(init)

X, Y = inputs()

total_loss = loss(X, Y)

train_op = train(total_loss)

coord = tf.train.Coordinator()

threads = tf.train.start_queue_runners(sess=sess, coord=coord)

training_steps = 100#实际训练迭代次数

for step in range(training_steps):#实际训练闭环

    sess.run([train_op])

    if step % 10 == 0:#查看训练过程损失递减

        print str(step)+ " loss: ", sess.run([total_loss])

print str(training_steps) + " final loss: ", sess.run([total_loss])

evaluate(sess, X, Y)#模型评估

import time

time.sleep(5)

coord.join(threads)

sess.close()

Tensorflow 线性回归

在有监督学习问题中, 线性回归是一种最简单的建模手段.

给定一个数据点集合作为训练集, 线性回归的目标是找到一个与这些数据最为吻合的线性函数.

y(x1,x2,x3,...,xk) = w1x1+w2x2+w3x3+...+wkxk+b

Y 为待预测值

X 是一组独立的预测变量

w 为权值 (可学习)

b 为偏移量 (可学习)

上面的公式用代码表示应该是

w=tf.Variable(tf.zeros([2,1]), name="weights")

b=tf.Variable(0.0, name="bias")

def inference(x):
    return tf.matmul(x,w)+b

接下来需要定义如何计算损失

对于这种简单的模型,将采用总平方误差,即每个训练样本产生的实际值与期望值之差的平方的总和

完整的例子如下

# -*- coding: utf-8 -*-
import tensorflow as tf

w=tf.Variable(tf.zeros([2,1]), name="weights")
b=tf.Variable(0.0, name="bias")


def inference(x):
    #计算推断模型在数据x上的输出,返回结果
    return tf.matmul(x,w)+b
    
def loss(x,y):
    #根据训练数据x及期望获得的结果y计算损失
    y_predicted = inference(x)
    return tf.reduce_sum(tf.squared_difference(y,y_predicted))
def inputs():
    #读取或生成训练数据x极其期望输出y
    weight_age = [[84,46],[73,20],[65,52],[70,30],[76,57],[69,25]]
    blood_fat_content = [354,190,405,263,451,302]
    return tf.to_float(weight_age),tf.to_float(blood_fat_content)
    
def train(total_loss):
    #根据计算的总损失训练或调整模型参数
    learning_rate = 0.00000001
    return tf.train.GradientDescentOptimizer(learning_rate).minimize(total_loss)
    
def evaluate(sess,x,y):
    #对训练得到的模型进行评估
    print sess.run(inference([[80.,25.]]))
    print sess.run(inference([[65.,25.]]))
    print sess.run(inference([[88.,33.]]))
    
saver = tf.train.Saver()

#在一个会话对象中启动数据流图, 搭建流程

with tf.Session() as sess:
    
    tf.initialize_all_variables().run()
    
    x,y = inputs()
    
    total_loss = loss(x,y)
    train_op = train(total_loss)
    
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(sess=sess,coord=coord)
    
    #实际的训练迭代次数
    training_steps = 100000
    
    for step in range(training_steps):
        sess.run([train_op])
        #输出损失递件过程
        if step%10==0:
            print "loss:", sess.run([total_loss])
            
        if step%1000==0:
            saver.save(sess,'my-checkpoint', global_step=step)
    
    evaluate(sess,x,y)
    
    coord.request_stop()
    coord.join(threads)
    
    saver.save(sess,'my-checkpoint', global_step=training_steps)
    sess.close()

这个例子运行时可以看到错误率一直在降低,最后我们尝试给出3个评估,也可以看到虽然输入数据非常少,但是33岁体重88公斤的人的血脂含量仍然是最大的

loss: [300929.16]
loss: [300928.09]
loss: [300927.0]
loss: [300925.97]
loss: [300924.88]
loss: [300923.78]
loss: [300922.69]
loss: [300921.62]
loss: [300920.56]
loss: [300919.5]
...

...

loss: [300392.31]
loss: [300392.34]
loss: [300392.31]
loss: [300392.31]
loss: [300392.34]
loss: [300392.31]
loss: [300392.31]
loss: [300392.31]
loss: [300392.31]
loss: [300392.31]
loss: [300392.31]
loss: [300392.31]
loss: [300392.31]
loss: [300392.28]
loss: [300392.31]
loss: [300392.31]
loss: [300392.28]
loss: [300392.31]
loss: [300392.28]
loss: [300392.28]
loss: [300392.31]
loss: [300392.31]
loss: [300392.31]
loss: [300392.31]
loss: [300392.25]
loss: [300392.28]
loss: [300392.31]
loss: [300392.28]
loss: [300392.28]
loss: [300392.25]
loss: [300392.25]
loss: [300392.28]
loss: [300392.25]
loss: [300392.25]
loss: [300392.25]
loss: [300392.25]
loss: [300392.25]
loss: [300392.25]
loss: [300392.25]
loss: [300392.25]
loss: [300392.25]
loss: [300392.22]
loss: [300392.25]
loss: [300392.22]
loss: [300392.25]
loss: [300392.25]
loss: [300392.25]
loss: [300392.22]
loss: [300392.22]
loss: [300392.19]
loss: [300392.19]
loss: [300392.19]
loss: [300392.19]
loss: [300392.22]
loss: [300392.19]
loss: [300392.19]
loss: [300392.19]
loss: [300392.19]
loss: [300392.19]
loss: [300392.19]
loss: [300392.19]
loss: [300392.19]
loss: [300392.19]
loss: [300392.19]
loss: [300392.19]
loss: [300392.19]
loss: [300392.16]
loss: [300392.16]
loss: [300392.16]
loss: [300392.19]
loss: [300392.16]
loss: [300392.19]
loss: [300392.16]
[[ 356.84619141]]
[[ 290.0894165]]
[[ 392.63818359]]
[root@localhost sample]#

为了看的清楚一点,我们对上面的代码稍作改动, 增加一些summary和基础数据

# -*- coding: utf-8 -*-
import tensorflow as tf

w=tf.Variable(tf.zeros([2,1]), name="weights")
b=tf.Variable(0.0, name="bias")


def inference(x):
    #计算推断模型在数据x上的输出,返回结果
    return tf.matmul(x,w)+b
    
def loss(x,y):
    #根据训练数据x及期望获得的结果y计算损失
    y_predicted = inference(x)
    return tf.reduce_sum(tf.squared_difference(y,y_predicted))
def inputs():
    #读取或生成训练数据x极其期望输出y
    weight_age = [[84,46],[73,20],[65,52],[70,30],[76,57],\
              [69,25],[63,28],[72,36],[79,57],[75,44],\
              [27,24],[89,31],[65,52],[57,23],[59,60],\
              [69,48],[60,34],[79,51],[75,50],[82,34],\
              [59,46],[67,23],[85,37],[55,40],[63,30]
             ]
    blood_fat_content = [354,190,405,263,451,302,288,385,402,365,209,290,346,254,395,434,220,374,308,220,311,181,274,303,244]
    return tf.to_float(weight_age),tf.to_float(blood_fat_content)
    
def train(total_loss):
    #根据计算的总损失训练或调整模型参数
    learning_rate = 0.00000001
    return tf.train.GradientDescentOptimizer(learning_rate).minimize(total_loss)
    
def evaluate(sess,x,y):
    #对训练得到的模型进行评估
    print sess.run(inference([[80.,25.]]))
    print sess.run(inference([[65.,25.]]))
    print sess.run(inference([[88.,33.]]))
    
saver = tf.train.Saver()

#在一个会话对象中启动数据流图, 搭建流程

with tf.Session() as sess:
    
    tf.initialize_all_variables().run()
    
    x,y = inputs()
    
    total_loss = loss(x,y)
    train_op = train(total_loss)
    
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(sess=sess,coord=coord)
    
    #实际的训练迭代次数
    training_steps = 100000000

    summary_writer = tf.train.SummaryWriter('./my_graph', sess.graph)
    tf.scalar_summary('total_loss', total_loss)
    sum_ops = tf.merge_all_summaries()
    
    for step in range(training_steps):
        sess.run([train_op])
        s_val = sess.run(sum_ops)
            summary_writer.add_summary(s_val, global_step=step)

        #输出损失递件过程
        if step%1000==0:
            print "loss:", sess.run([total_loss])," step:",step
            
        if step==(training_steps-1):
            print "w:", sess.run(w)
            print "b:", sess.run(b)

        #if step%1000==0:
            #saver.save(sess,'my-checkpoint', global_step=step)
    
    evaluate(sess,x,y)
    
    coord.request_stop()
    coord.join(threads)
    
    saver.save(sess,'my-checkpoint', global_step=training_steps)
    sess.close()

可以适当调整

learning_rate = 0.0000002

training_steps = 1000000

来提高精度

这里我是对loss进行统计,所以可以看到随着step递增,loss逐渐减少

总结:

其实一开始我对下面的语句是有很大疑问的

def inference(x):
    return tf.matmul(x,w)+b

首先这里 x 变量是一个矩阵, w 也是一个矩阵, 矩阵与矩阵的乘积应该也是一个矩阵

但是这里的 b变量 是一个常数, 理论上矩阵和常数相加, 必需先把常数转换成矩阵,结果也应该是一个矩阵,所以我蒙圈了

只能去查函数说明

tf.matmul(a, b, transpose_a=False, transpose_b=False, a_is_sparse=False, b_is_sparse=False, name=None)

Multiplies matrix a by matrix b, producing a * b.

The inputs must be two-dimensional matrices, with matching inner dimensions, possibly after transposition.

Both matrices must be of the same type. The supported types are: floatdoubleint32complex64.

Either matrix can be transposed on the fly by setting the corresponding flag to True. This is False by default.

If one or both of the matrices contain a lot of zeros, a more efficient multiplication algorithm can be used by setting the corresponding a_is_sparse or b_is_sparse flag to True. These are False by default.

For example:

# 2-D tensor `a`
a = tf.constant([1, 2, 3, 4, 5, 6], shape=[2, 3]) => [[1. 2. 3.]
                                                      [4. 5. 6.]]
# 2-D tensor `b`
b = tf.constant([7, 8, 9, 10, 11, 12], shape=[3, 2]) => [[7. 8.]
                                                         [9. 10.]
                                                         [11. 12.]]
c = tf.matmul(a, b) => [[58 64]
                        [139 154]]

也就是

1*7+2*9+3*11  1*8+2*10+3*12
4*7+5*9+6*11  4*8+5*10+6*12

最后我推荐你自己动手试试它的这些函数是如何输出的

# -*- coding: utf-8 -*-
import tensorflow as tf

a = [[1.,2.],[3.,4.]]
b = [[1.],[2.]]
c = tf.Variable(2.1, name="bias")

with tf.Session() as sess:
    tf.initialize_all_variables().run()
    print(sess.run(tf.to_float(a)))
    print(sess.run(tf.to_float(b)))
    print(sess.run(tf.matmul(a,b)))
    print(sess.run(tf.matmul(a,b)+tf.to_float(c)))
    print(sess.run(tf.reduce_sum(tf.squared_difference(tf.matmul(a,b),tf.matmul(a,b)+tf.to_float(c)))))

结果是

[root@localhost sample]# python sample2.py 
[[ 1.  2.]
 [ 3.  4.]]
[[ 1.]
 [ 2.]]
[[  5.]
 [ 11.]]
[[  7.0999999 ]
 [ 13.10000038]]
8.82

从行为上来看

这里矩阵与常数相加应该和matlab上差不多, 每个子都被加上了这个常量