LoRA Fine-tune a Text Classifier

You're working with a pretrained transformer that you want to adapt to a new 3-class classification task. Full fine-tuning would update all 25K+ parameters, but you only have a small dataset and limited compute. LoRA lets you freeze the base model and train only a small pair of low-rank matrices (A and B) per weight, reducing trainable parameters to under 3% of the original while achieving competitive accuracy. The provided SimpleTransformer is a minimal 2-layer model with frozen weights. Your job is to implement the LoRAAdapter class, wire it into the forward pass, and build the training loop.

1. 1.Initialize LoRA matrices A (Gaussian) and B (zeros) with correct shapes and compute the scaling factor alpha/rank
2. 2.Implement the LoRA forward pass: (x @ A @ B) * scaling
3. 3.Implement delta_W property that returns the full weight update A @ B * scaling
4. 4.Implement train_step that applies LoRA to the attention weight and computes the loss
5. 5.Implement evaluate that computes classification accuracy with LoRA-adapted weights