devbox
煮酒与君饮
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

【源码复现】图神经网络之vanilla-GCN_vanilla gcn

作者：煮酒与君饮 | 2024-06-28 20:50:25

赞

踩

vanilla gcn

一、论文简介

二、论文核心简述

 本文作者从谱图卷积的角度出发，首先提出了在傅里叶域上对于图上信号 x x x的图卷积操作，然而传统的图卷积操作，采用矩阵乘积的形式，计算量大，而且，对于庞大的图结构，计算图拉普拉斯矩阵的特征值分解，计算复杂度大。为了避免巨大的计算复杂度开销，作者采用一阶切比雪夫多项式来巧妙地近似，并采用重整正则化的技巧，得到最后的分层结构的图卷积神经网络，GCN的分层传播规则如下：

H ( l + 1 ) = σ ( A ^ H ( l ) W ( l ) ) H^{(l+1)} = \sigma(\hat AH^{(l)}W^{(l)}) H(l+1)=σ(A^H(l)W(l))
其中 A ^ = D ~ − 1 2 A ~ D ~ − 1 2 \hat A= \tilde D^{-\frac{1}{2}} \tilde A \tilde D^{-\frac{1}{2}} A^=D~−21​A~D~−21​， A ~ \tilde A A~为带有自环的邻接矩阵，即 A ~ = A + I \tilde A =A+I A~=A+I， D ~ \tilde D D~为 A ~ \tilde A A~的度对角矩阵； W ( l ) W^{(l)} W(l)是第 l l l层可学习的线性变换矩阵； H ( l ) H^{(l)} H(l)是第 l l l层的输出结果， H ( 0 ) = X H^{(0)}=X H(0)=X； σ ( . ) \sigma(.) σ(.)为非线性激活函数 R e L U ReLU ReLU。
 在GCN的每个图卷积层，结点特征主要按照以下三种方式更新：

  • 特征传播
  • 线性变换
  • 非线性激活

特征传播：每个结点的特征来自当前结点特征和自己邻域结点特征加权之和，类似于一种局部平滑操作。
线性变化和非线性激活：类似于传统的MLP，对于平滑后的矩阵进行线性变化和非线性激活操作。
 最后一层的结点分类采用 s o f t m a x softmax softmax分类器。最终得到的GCN模型如下：
Z = f ( X , A ) = s o f t m a x ( A ^    R e L U ( A ^ X W ( 0 ) ) W ( 1 ) ) Z=f(X,A)=softmax(\hat A \; ReLU(\hat AXW^{(0)})W^{(1)}) Z=f(X,A)=softmax(A^ReLU(A^XW(0))W(1))

三、源码

1、utils.py

import pandas as pd
import numpy as np
import torch

def load_data(filename = 'cora'):

    dataset = pd.read_csv('./data/{}.content'.format(filename),sep='\t', header=None, low_memory=False)
    #结点数量
    node_num = dataset.shape[0]
    index_list = list(dataset.index)
    #数据集中
    id_list = list(dataset[0])

    #建立论文编号到重新索引的字典
    id_to_index = dict(zip(id_list,index_list))

    #从第一列到倒数第二列定义为结点的特征向量
    #data.iloc  取某行某列数据
    feature = np.array(dataset.iloc[:,1:-1])

    str_label = np.array(dataset.iloc[:,-1])
    st = set()
    for str in str_label:
        st.add(str)
    #将字符str标签转化为[0~6]标签
    label_to_num = dict(zip(st,range(len(st))))
    #0~6
    labels = [label_to_num[i] for i in str_label]

    #读取结点间的引用关系
    data_cites = pd.read_csv(f'./data/{filename}.cites', sep='\t', header=None)
    #首先创建零矩阵
    adj_matrix = np.zeros((node_num,node_num))
    #构建邻接矩阵
    for node,cite_node in zip(data_cites[0],data_cites[1]):
        node_idx,cite_node_idx = id_to_index[node],id_to_index[cite_node]
        adj_matrix[node_idx][cite_node_idx] = adj_matrix[cite_node_idx][node_idx] =1

    #正则化
    adj_matrix = normlization(adj_matrix)
    #print(adj_matrix)

    #转化为torch可操作的类型
    adj_matrix = torch.FloatTensor(adj_matrix)
    feature = torch.FloatTensor(feature)
    labels = torch.LongTensor(labels)

    return feature,labels , adj_matrix
#对邻接矩阵重整正则化
def normlization(adj):
    adj = adj + np.eye(adj.shape[0])
    #对每一行求和并拉平
    row_sum = adj.sum(1).flatten()
    #对每个元素求a**(-0.5)
    D_inv_sqrt = np.power(row_sum,-0.5)
    #将无穷值转化为0
    D_inv_sqrt[np.isinf(D_inv_sqrt)] = 0
    #转换为对角矩阵
    D_mat_inv_sqrt =np.diag(D_inv_sqrt)
    #D~^(-0.5)*A~*D~(-0.5)
    return D_mat_inv_sqrt.dot(adj).dot(D_mat_inv_sqrt)
#计算ACC
def accuracy(output, labels):
    
    preds = output.max(1)[1].type_as(labels)
    correct = preds.eq(labels).double()
    correct = correct.sum()
    return correct / len(labels)

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56
  • 57
  • 58
  • 59
  • 60
  • 61
  • 62
  • 63
  • 64
  • 65
  • 66
  • 67
  • 68

2、model.py

import torch 
from torch import nn
from torch.nn import Module
from torch.nn import Parameter


class GraphConvolution(Module):
    def __init__(self,input_dim,output_dim):
        super(GraphConvolution,self).__init__()
        self.input_feature = input_dim
        self.output_feature = output_dim
    
        self.weight = Parameter(torch.empty(size=(input_dim,output_dim)))
        self.init_param()
	def init_param(self):
		nn.init.xavier_uniform_(self.weight.data,gain=1.414)
    def forward(self,feature,adj):
    	#特征传播
        output = torch.mm(adj,feature)
        #线性变化
        output = torch.mm(output,self.weight)
        return output
class GCN(Module):
    def __init__(self,input_dim,hid_dim,output_dim,dropout):
        super(GCN,self).__init__()
        self.layer1 = GraphConvolution(input_dim,hid_dim)
        self.layer2 = GraphConvolution(hid_dim,output_dim)
        self.dropout = dropout

    def forward(self,feature,adj):
    	#第一层
        output = self.layer1(feature,adj)
        #非线性激活
        output = F.relu_(output)
        #output = F.relu(self.layer1(feature,adj))
        #dropout
        if self.dropout is not None:
            output = F.dropout(output,self.dropout,self.training)
        #第二层
        output = self.layer2(output,adj)
        #预测结果
        return F.log_softmax(output,dim=1)

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42

3、train.py

import torch
import os
from torch.optim import Adam
from torch.nn import functional as F
import numpy as np
from model import GCN
from utils import load_data,accuracy
os.environ['KMP_DUPLICATE_LIB_OK'] = 'TRUE'

#超参数配置
epochs = 100
lr = 0.01
hid_dim = 16#32
weight_decay = 5e-4
dropout = 0.5
#device ='cpu'#'cuda' if torch.cuda.is_available() else 'cpu'
dataset_name = 'cora'
features,labels , adj_matrix = load_data(dataset_name)
input_dim = features.shape[1]
output_dim = 7
#加载模型
model = GCN(input_dim,hid_dim,output_dim,dropout)
#数据集划分
idx_train = range(140)
idx_valid = range(300,300+500)
idx_test = range(1708,2708)
idx_train = torch.LongTensor(idx_train)
idx_valid = torch.LongTensor(idx_valid)
idx_test = torch.LongTensor(idx_test)
#优化器
opt = Adam(model.parameters(),lr=lr,weight_decay=weight_decay)
def train(model,opt,epochs,best_acc):
    model.train()
    for epoch in range(epochs):
        output = model(features,adj_matrix)
        '''
        F.nll_loss计算方式是下式，在函数内部不含有提前使用softmax转化的部分；
        nn.CrossEntropyLoss内部先将输出使用softmax方式转化为概率的形式，后使用F.nll_loss函数计算交叉熵。
        '''
        opt.zero_grad()
        loss_train = F.nll_loss(output[idx_train],labels[idx_train])
        acc_train = accuracy(output[idx_train],labels[idx_train])
        loss_train.backward()
        opt.step()
        loss_val = F.nll_loss(output[idx_valid],labels[idx_valid])
        acc_val = accuracy(output[idx_valid],labels[idx_valid])
        
        print(f"Epoch {epoch}, train loss = {loss_train:.4f},train accuracy = {acc_train:.4f}")
        print(f"valid loss = {loss_val:.4f},valid accuracy = {acc_val:.4f}")   
def test(model,features,adj):
    model.eval()
    model.to(device)
    features.to(device)
    adj.to(device)
    output = model(features,adj)
    loss_test = F.nll_loss(output[idx_test], labels[idx_test])
    acc_test = accuracy(output[idx_test], labels[idx_test])
    print(f"test loss = {loss_test:.4f},test accuracy = {acc_test:.4f}")
train(model,opt,epochs)
test(model,features,adj_matrix)

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56
  • 57
  • 58
  • 59
  • 60

4、DGL复现

这里只列出了模型

import os
os.environ["DGLBACKEND"] = "pytorch"
import dgl
import dgl.function as fn
import torch 
import torch.nn as nn
import torch.nn.functional as F
from dgl import DGLGraph

class GCNLayer(nn.Module):
    def __init__(self,infeat,outfeat) -> None:
        super(GCNLayer,self).__init__()
        self.layer = nn.Linear(infeat,outfeat,bias=False)
        self._in_feats = infeat
        self._out_feats = outfeat
        self.weight = nn.Parameter(torch.Tensor(infeat, outfeat))
        nn.init.xavier_uniform_(self.weight)
    def forward(self,g,feature):
        with g.local_scope():
            g = dgl.add_self_loop(g)
            feat_src,feat_dst = dgl.utils.expand_as_pair(feature, g)
            degs = g.out_degrees().to(feat_src).clamp(min=1)
            norm = torch.pow(degs,-0.5)
            shp = norm.shape + (1,) * (feat_src.dim() -1 )
            norm = torch.reshape(norm,shp)

            feat_src = feat_src * norm
            feat_src = torch.matmul(feat_src, self.weight)
            g.srcdata["u"] = feat_src
            g.dstdata['v'] = feat_dst
            g.update_all(
                fn.copy_u("u", "m"), 
                fn.sum(msg="m", out="h")
                )
            rst = g.ndata['h']

            degs = g.in_degrees().to(feat_dst).clamp(min=1)
            norm = torch.pow(degs, -0.5)
            shp = norm.shape + (1,) * (feat_dst.dim() - 1)
            norm = torch.reshape(norm, shp)
            rst = rst * norm

            return rst

class GCN(nn.Module):
    def __init__(self,infeat,hidfeat,outfeat ,dropout) -> None:
        super(GCN,self).__init__()
        self.dropout = dropout
        self.layer1 = GCNLayer(infeat,hidfeat)
        self.layer2 = GCNLayer(hidfeat,outfeat)
    
    def forward(self,g,features):
        x = F.dropout(features,self.dropout,training=self.training)
        x = F.relu(self.layer1(g,x))
        #print(x.shape)
        x = F.dropout(x,self.dropout,training=self.training)
        x = self.layer2(g,x)
        return F.log_softmax(x,dim=1)
    

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56
  • 57
  • 58
  • 59
声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/煮酒与君饮/article/detail/767231
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号

        
cppcmd=keepalive&