Comments (4)
F.softmax(torch.matmul(
linear_query,
linear_key.transpose(-2, -1)
) / math.sqrt(self.__hidden_dim), dim=-1)
是不是应该这样呢??
from gl-gin.
@jingyonglin 您好,你说的是对的,我们实验过,影响不大,之前follow的stack-propagation也是这么写的,我们就保持一致进行公平比较了。
from gl-gin.
@jingyonglin 您好,你说的是对的,我们实验过,影响不大,之前follow的stack-propagation也是这么写的,我们就保持一致进行公平比较了。
嗯, 我现在也正准备开始试跑 stack propagation, 关于这个公式顺序的影响我也没试验过, 我也自己试试呗, 我觉得当然计算上是没问题能正常运行的, 不过我认为这样模型层面的公式改动还是值得记录在paper中的, 提出来这个疑问也是, 如果不仔细看真的就忽略过去了
from gl-gin.
@jingyonglin 非常谢谢你的意见,更多的交流可以给我发邮件[email protected]。或者微信沟通:)
from gl-gin.
Related Issues (12)
- 关于MixSNIPS性能问题 HOT 1
- 想确认一下数据 HOT 2
- 关于中文意图识别与槽位填充的问题,slot值在原文未出现该如何标注 HOT 2
- 关于论文中metrics的问题 HOT 4
- mixatis数据atis_day_name类别在train中未出现 HOT 1
- More information about t_test HOT 1
- generate_global_adj_gat函数的问题
- LSTM object has no attribute _flat_weights
- code run error
- Optimal hyper-parameters HOT 4
- bert预训练模型
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gl-gin.