Comments (1)
Position Embedding 中我觉得元素打乱顺序应该没什么关系,作者的代码中将经过 cos 的 sin 的数值分别放在一起应该如果是 concat 模式的话感觉对结果是没什么影响,如果是 sum 的话就不好说了。
PS: 这个层好像写的有点问题,实际运行时,在训练集上达到一定的准确率后就上不去了,永远训练不出来(不会过拟合)。
from attention.
Related Issues (20)
- attention_keras.py中,Position_Embedding的pos的计算 HOT 2
- tf中position HOT 4
- 请问一下,tf版的78,80行的转置操作是什么意思呢? 去掉这两行进行训练对结果也没什么影响。 HOT 2
- attention_keras.py
- 请问Q,K,V分别是从哪里输入的呢? HOT 2
- K,Q,V映射
- 模型加载问题 HOT 2
- I have no idea what method "to_mask" helps. HOT 1
- Is there tensorflow implementation of sparse attention? HOT 3
- a = K.permute_dimensions(a, (0, 3, 2, 1)) HOT 2
- Position_Embedding肯能有些问题 HOT 1
- Keras的batch_dot()如何用pytorch改写 HOT 2
- 你好,这个注意力有没有使用的例子呢?如何调用注意力层呢? HOT 1
- K.permute_dimensions(A, (0,3,2,1))运行时报错 HOT 13
- position_ij = tf.matmul(position_i, position_j) 应该用 multiply 而不是 matmul HOT 1
- How to understand the decoder input(target)? HOT 1
- 不支持mask HOT 2
- 感觉好像少了参数Wo HOT 1
- 行为识别 用attention HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from attention.