Image: Positional encoding
Size of this preview: 800 × 400 pixels. Other resolutions: 320 × 160 pixels | 1,600 × 800 pixels.
Original image (1,600 × 800 pixels, file size: 151 KB, MIME type: image/png)
Description: Plotted in Julia. using CairoMakie function f(pos, N, k, d) theta = pos/(N^(2*k/d)) if k % 2 ==1 return sin(theta) else return cos(theta) end end ks = 0:64 poss = 0:1000 data = [f(pos, 10000, k, 100) for pos in poss, k in ks]; fig = Figure(resolution=(1600, 800)) ax = Axis(fig[1, 1]) Makie.heatmap!(data', color=:viridis) ax.title="Positional encoding" ax.xlabel="vector index" ax.ylabel="position" save("positional encoding.png", fig)
Title: Positional encoding
Credit: Own work
Author: Cosmia Nebula
Usage Terms: Creative Commons Attribution-Share Alike 4.0
License: CC BY-SA 4.0
License Link: https://creativecommons.org/licenses/by-sa/4.0
Attribution Required?: Yes
Image usage
The following page links to this image:
All content from Kiddle encyclopedia articles (including the article images and facts) can be freely used under Attribution-ShareAlike license, unless stated otherwise.
