Building a transformer without training
I started by learning how a GPT works from scratch, training small models on my laptop and asking what they were actually good for. Every idea I tried, from Sigma rules to ATT&CK mapping, ran into the same question: couldn't normal code do this anyway? So I flipped it. Base64 is already done correctly by code, so could a transformer do the same thing exactly, with no training at all? I wrote one manually to find out, partly to learn, and partly because I hate waiting when the GPU is already busy training my other projects. 01001000 01100101 01101100 01101100 01101111 00100000 01000100 01100001 01110010 01110000 01100001 01101110 00100001 01001000 01100101 01101100 01101100 01101111 00100000 01000100 01100001 01110010 01110000 01100001 Hello Darpan! ↓ SGVsbG8gRGFycGFuIQ== That became qkvc. You describe an algorithm in a short file, and its builder makes a transformer that runs it. Every weight is placed by rule and every number has a name. The target I'll use my own text, He...