Skip to content

Commit

Permalink
Merge pull request karpathy#142 from kovkev/patch-1
Browse files Browse the repository at this point in the history
Fix the position of a comma
  • Loading branch information
karpathy authored Apr 13, 2023
2 parents ea24604 + c253115 commit ad62003
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion model.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ def forward(self, x):
B, T, C = x.size() # batch size, sequence length, embedding dimensionality (n_embd)

# calculate query, key, values for all heads in batch and move head forward to be the batch dim
q, k ,v = self.c_attn(x).split(self.n_embd, dim=2)
q, k, v = self.c_attn(x).split(self.n_embd, dim=2)
k = k.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)
q = q.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)
v = v.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)
Expand Down

0 comments on commit ad62003

Please sign in to comment.