Qwen Coder 2.5 32b on M4 Max
✎ Published: 2024-11-11 10:30:30
Initial notes on how Qwen Coder 2.5 performs on a Macbook Pro M4 Max
I received my new Macbook Pro M4 Max today and at the same time Qwen released their new
Coder 2.5 model.
I quickly installed LMStudio, downloaded the GGUF models and ran some tests.
I used two different GGUF models for these tests:
- Qwen Coder 2.5 32b q8
- Qwen Coder 2.5 32b q2_k
I chose these two to observer the difference in performance between lightly and heavily quantized models.
Environment
Those were loaded with the following settings:
- Context Length: 4096k
- Flash Attention: Disabled
- GPU Offload: Full
- CPU Thread Pool Size: 12
- Evaluation Batch Size 512
- Keep Model in Memory: True
- Try mmap: True
First Prompt
Results on this prompt (48 tokens):
I have this react fn in typescript
export function LoginForm()
how can I change it so that it can receive these two functions as parameters:
export async function signup(formData: FormData) {}
export async function login(formData: FormData) {}
(Note that the prompt is deliberately not well written)
q8
- Response: 12.96 tok/sec, 564 tokens, 1.02s to first token
Response
To modify your `LoginForm` component to accept the `signup` and `login` functions as parameters, you'll need to define an interface for the props that the component will receive. This ensures type safety in TypeScript.
Here's how you can do it:
- Define a
LoginFormProps
interface that includes the types for the signup
and login
functions.
- Use these props within your
LoginForm
component.
Here is the updated code:
import React from 'react';
// Define the interface for the LoginForm props
interface LoginFormProps {
signup: (formData: FormData) => Promise<void>;
login: (formData: FormData) => Promise<void>;
}
export function LoginForm({ signup, login }: LoginFormProps) {
// Example usage of the signup and login functions within the component
const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
event.preventDefault();
const formData = new FormData(event.currentTarget);
// Determine if it's a signup or login attempt
// This is just an example, you might want to use another mechanism to determine this.
const isSignupAttempt = true; // Change this based on your logic
try {
if (isSignupAttempt) {
await signup(formData);
} else {
await login(formData);
}
} catch (error) {
console.error('Error during form submission:', error);
}
};
return (
<form onSubmit={handleSubmit}>
{/* Your form fields go here */}
<button type="submit">Submit</button>
</form>
);
}
// Example usage of LoginForm component
export async function signup(formData: FormData) {
// Signup logic here
console.log('Signing up with:', formData);
}
export async function login(formData: FormData) {
// Login logic here
console.log('Logging in with:', formData);
}
In this example:
LoginFormProps
defines the shape of the props that LoginForm
will receive, including the types for the signup
and login
functions.
- The
handleSubmit
function is an example of how you might use these functions within the component. It determines whether to call signup
or login
based on some condition (isSignupAttempt
) which you should adjust according to your specific logic.
- The form element uses the
onSubmit
event handler to trigger the handleSubmit
function when the form is submitted.
You can now pass the signup
and login
functions as props when rendering the LoginForm
component:
<LoginForm signup={signup} login={login} />
This ensures that your component is flexible and can handle different implementations of the signup
and login
functions.
q2_k
- Response: 17.95 tok/sec, 494 tokens, 0.76s to first token
Response
To modify your `LoginForm` component to accept the `signup` and `login` functions as parameters, you need to define a TypeScript interface or type for the props that your component will receive. This allows you to specify that the component should expect these two functions as part of its props.
Here's how you can do it:
- Define an interface for the props.
- Use this interface in the
LoginForm
function definition.
Here is the updated code:
import React from 'react';
// Define the type for the props that LoginForm will receive
interface LoginFormProps {
signup: (formData: FormData) => Promise<void>;
login: (formData: FormData) => Promise<void>;
}
export function LoginForm({ signup, login }: LoginFormProps) {
// Your form rendering logic here
const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
event.preventDefault();
const formData = new FormData(event.currentTarget);
// Example of calling the passed functions
if (someCondition) { // Replace with your actual condition to decide between signup and login
await signup(formData);
} else {
await login(formData);
}
};
return (
<form onSubmit={handleSubmit}>
{/* Your form fields here */}
<button type="submit">Submit</button>
</form>
);
}
// Example usage of LoginForm
// const App = () => {
// const handleSignup = async (formData: FormData) => {
// // signup logic
// };
// const handleLogin = async (formData: FormData) => {
// // login logic
// };
// return <LoginForm signup={handleSignup} login={handleLogin} />;
// };
In this example:
LoginFormProps
is an interface that defines the shape of the props expected by LoginForm
. It includes two properties: signup
and login
, each being a function that takes a FormData
object and returns a Promise<void>
.
- The
LoginForm
component deconstructs the props
to receive signup
and login
directly.
- A sample form handling logic is included, where you can determine which function (
signup
or login
) should be called based on some condition.
You can then pass the signup
and login
functions when using the LoginForm
component, as shown in the commented example usage.
Quality
If you open the responses, you'll see that the quality at q2_k really degrades. At the same time, the time difference of 12 to 17 seconds is not that long.
Second Prompt
Please find bugs in the following typescript code and explain in detail what the bug is and provide a git patch with a fix:
import React from 'react';
// Define the interface for the LoginForm props
interface LoginFormProps {
signup: (formData: FormData) => Promise<void>;
login: (formData: FormData) => Promise<void>;
}
export function LoginForm({ signup, login }: LoginFormProps) {
// Example usage of the signup and login functions within the component
const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
event.preventDefault();
const formData = new FormData(event.currentTarget);
// Determine if it's a signup or login attempt
// This is just an example, you might want to use another mechanism to determine this.
const isSignupAttempt = true; // Change this based on your logic
try {
if (isSignupAttempt) {
await signup(formData);
} else {
await signup(formData);
}
} catch (error) {
console.error('Error during form submission:', error);
}
};
return (
<form onSubmit={handleSubmit}>
{/* Your form fields go here */}
<button type="submitt">Submit</button>
</form>
);
}
// Example usage of LoginForm component
export async function signup(formData: FormData) {
// Signup logic here
console.log('Signing up with:', formData);
}
export async function login(formData: FormData) {
// Login logic here
console.log('Logging in with:', formData);
}
Results
q8
- Response: 12.72 tok/sec, 536 tokens, 2.20s to first token
Response
### Bugs Identified:
-
Incorrect Function Call:
- In the
handleSubmit
function, when isSignupAttempt
is false
, it incorrectly calls signup(formData)
instead of login(formData)
.
-
Typo in Button Type Attribute:
- The
<button>
element has a typo in its type
attribute: submitt
should be submit
.
Detailed Explanation:
-
Incorrect Function Call:
- This bug will cause the application to always attempt to sign up, even if the intent is to log in. When
isSignupAttempt
is false
, the code should call the login
function instead of signup
.
-
Typo in Button Type Attribute:
- The typo in the button type means that clicking the button will not trigger the form's
onSubmit
event handler as intended. The correct value for a submit button is submit
.
Git Patch with Fixes:
Here is the patch file (0001-Fix-signup-login-call-and-typo-in-button-type.patch
) to address these issues.
diff --git a/src/LoginForm.tsx b/src/LoginForm.tsx
index 1234567..89abcde 100644
@@ -20,7 +20,7 @@ export function LoginForm({ signup, login }: LoginFormProps) {
if (isSignupAttempt) {
await signup(formData);
} else {
- await signup(formData);
+ await login(formData);
}
} catch (error) {
console.error('Error during form submission:', error);
@@ -34,7 +34,7 @@ export function LoginForm({ signup, login }: LoginFormProps) {
</form>
);
}
-
+
// Example usage of LoginForm component
export async function signup(formData: FormData) {
// Signup logic here
Applying the Patch:
- Save the above content into a file named
0001-Fix-signup-login-call-and-typo-in-button-type.patch
.
- Navigate to your project directory in the terminal.
- Apply the patch using the
git apply
command:git apply 0001-Fix-signup-login-call-and-typo-in-button-type.patch
After applying the patch, the code should work correctly by calling the appropriate function based on the value of isSignupAttempt
, and the button will properly submit the form.
q2_k
- Response: 17.95 tok/sec, 453 tokens, 2.08s to first token
Response
The provided TypeScript code contains a few issues that need to be addressed. Let's go through each bug and provide the necessary fixes.
Bugs Identified:
- Incorrect Function Call: The
handleSubmit
function incorrectly calls the signup
function twice instead of calling login
for login attempts.
- Typo in Button Type Attribute: The button type is misspelled as "submitt" instead of "submit".
Let's address these issues with a git patch.
Git Patch
diff --git a/src/LoginForm.tsx b/src/LoginForm.tsx
index 1234567..89abcde 100644
@@ -18,7 +18,7 @@ export function LoginForm({ signup, login }: LoginFormProps) {
try {
if (isSignupAttempt) {
await signup(formData);
- } else {
- await signup(formData);
+ } else {
+ await login(formData);
}
} catch (error) {
console.error('Error during form submission:', error);
@@ -30,7 +30,7 @@ export function LoginForm({ signup, login }: LoginFormProps) {
{/* Your form fields go here */}
- <button type="submitt">Submit</button>
+ <button type="submit">Submit</button>
</form>
);
}
Explanation of Fixes:
- Correct Function Call: The
handleSubmit
function now correctly calls the login
function when it is not a signup attempt.- await signup(formData);
+ await login(formData);
- Button Type Correction: The button type attribute was corrected from "submitt" to "submit".
- <button type="submitt">Submit</button>
+ <button type="submit">Submit</button>
By applying this patch, the LoginForm
component should work as intended, calling either the signup
or login
function based on the form submission context and correctly handling button submissions.
Parameters
I also wanted to try the same prompt with different model settings. So here the prompt is as follows:
Can you explain what this code does and then convert it to swiftui?
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>3D Rotating Cube</title>
<style>
body {
display: flex;
justify-content: center;
align-items: center;
height: 100vh;
margin: 0;
background-color: #f0f0f0;
}
canvas {
background-color: #ffffff;
}
</style>
</head>
<body>
<canvas id="cubeCanvas" width="500" height="500"></canvas>
<script>
const canvas = document.getElementById("cubeCanvas");
const ctx = canvas.getContext("2d");
const width = canvas.width;
const height = canvas.height;
const halfWidth = width / 2;
const halfHeight = height / 2;
let angleX = 0;
let angleY = 0;
function drawCube() {
ctx.clearRect(0, 0, width, height);
const vertices = [
[-1, -1, -1], [1, -1, -1], [1, 1, -1], [-1, 1, -1],
[-1, -1, 1], [1, -1, 1], [1, 1, 1], [-1, 1, 1]
];
const edges = [
[0, 1], [1, 2], [2, 3], [3, 0],
[4, 5], [5, 6], [6, 7], [7, 4],
[0, 4], [1, 5], [2, 6], [3, 7]
];
const faces = [
[0, 1, 2, 3], [4, 5, 6, 7], [0, 1, 5, 4],
[2, 3, 7, 6], [1, 2, 6, 5], [0, 3, 7, 4]
];
const projectedVertices = vertices.map(v => {
let [x, y, z] = v;
let tempY = y * Math.cos(angleX) - z * Math.sin(angleX);
let tempZ = y * Math.sin(angleX) + z * Math.cos(angleX);
y = tempY;
z = tempZ;
let tempX = x * Math.cos(angleY) - z * Math.sin(angleY);
z = x * Math.sin(angleY) + z * Math.cos(angleY);
x = tempX;
const scale = 200 / (z + 4);
const projX = x * scale + halfWidth;
const projY = y * scale + halfHeight;
return [projX, projY];
});
ctx.fillStyle = "rgba(173, 216, 230, 0.5)"; faces.forEach(face => {
ctx.beginPath();
ctx.moveTo(...projectedVertices[face[0]]);
for (let i = 1; i < face.length; i++) {
ctx.lineTo(...projectedVertices[face[i]]);
}
ctx.closePath();
ctx.fill();
});
ctx.strokeStyle = "darkblue";
ctx.lineWidth = 2;
edges.forEach(edge => {
const [start, end] = edge;
ctx.beginPath();
ctx.moveTo(...projectedVertices[start]);
ctx.lineTo(...projectedVertices[end]);
ctx.stroke();
});
}
function animate() {
angleX += 0.01;
angleY += 0.01;
drawCube();
requestAnimationFrame(animate);
}
animate();
</script>
</body>
</html>
Results
q2_k
- ~8k context, Enable Flash Attention
- 16.92 tok/sec, 859 tokens, 5.45s to first token
- Result didn't compile in Xcode, has 4 errors which are probably easy to fix
Response
import SwiftUI
import SceneKit
struct ContentView: View {
@State private var angleX: Float = 0.0
@State private var angleY: Float = 0.0
var body: some View {
VStack {
SCNVeviewContainer(angleX: $angleX, angleY: $angleY)
.frame(width: 500, height: 500)
Spacer()
}
.onAppear {
let _ = Timer.scheduling(timerTick, interval: 0.016) // Approximately 60 FPS
}
}
func timerTick(_ timer: Timer) {
angleX += 0.01
angleY += 0.01
}
}
struct SCNVeviewContainer: UIViewRepresentable {
@Binding var angleX: Float
@Binding var angleY: Float
let sceneView = SCNView()
func makeUIView(context: Context) -> SCNView {
// Set the background color of the scene view
sceneView.backgroundColor = UIColor.white
// Create a new SCNScene
let scene = SCNScene()
// Create a cube geometry
let cubeGeometry = SCNBox(width: 1.0, height: 1.0, length: 1.0, chamferRadius: 0)
cubeGeometry.firstMaterial?.diffuse.contents = UIColor(red: 0.68, green: 0.85, blue: 0.92, alpha: 0.5) // Light blue with transparency
// Create a node and add the geometry to it
let node = SCNNode(geometry: cubeGeometry)
// Set the initial position of the node
node.position = SCNVector3(0, 0, -4)
// Add the node to the scene
scene.rootNode.addChildNode(node)
// Set the scene to the view
sceneView.scene = scene
return sceneView
}
func updateUIView(_ uiView: SCNView, context: Context) {
let rotationX = SCNRotation(x: 1, y: 0, z: 0, angle: angleX)
let rotationY = SCNRotation(x: 0, y: 1, z: 0, angle: angleY)
// Apply the rotations to the node
uiView.scene!.rootNode.enumerateChildNodes { (node, stop) in
node.rotation = rotationX * rotationY
}
}
}
@main
struct ContentViewApp: App {
var body: some Scene {
WindowGroup {
ContentView()
.frame(width: 500, height: 500)
.background(Color(.lightGray))
}
}
}
q2_k
- full context, Disable Flash Attention
- 15.22 tok/sec, 990 tokens, 6.95s to first token
- Result didn't compile in Xcode, has 1 error (wrong pattern matching) which should be easy to fix
Response
import SwiftUI
struct ContentView: View {
@State private var angleX: Double = 0
@State private var angleY: Double = 0
let vertices: [[Double]] = [
[-1, -1, -1], [1, -1, -1], [1, 1, -1], [-1, 1, -1],
[-1, -1, 1], [1, -1, 1], [1, 1, 1], [-1, 1, 1]
]
let edges: [[Int]] = [
[0, 1], [1, 2], [2, 3], [3, 0],
[4, 5], [5, 6], [6, 7], [7, 4],
[0, 4], [1, 5], [2, 6], [3, 7]
]
let faces: [[Int]] = [
[0, 1, 2, 3], [4, 5, 6, 7], [0, 1, 5, 4],
[2, 3, 7, 6], [1, 2, 6, 5], [0, 3, 7, 4]
]
var body: some View {
GeometryReader { geometry in
Path { path in
let width = geometry.size.width
let height = geometry.size.height
let halfWidth = width / 2
let halfHeight = height / 2
let projectedVertices = vertices.map { v in
let [x, y, z] = v
// Rotate around X axis
let tempY = y * cos(angleX) - z * sin(angleX)
let tempZ = y * sin(angleX) + z * cos(angleX)
// Rotate around Y axis
let tempX = x * cos(angleY) - tempZ * sin(angleY)
let finalZ = x * sin(angleY) + tempZ * cos(angleY)
// Project the 3D coordinates to 2D
let scale = 200 / (finalZ + 4)
let projX = tempX * scale + halfWidth
let projY = tempY * scale + halfHeight
return CGPoint(x: projX, y: projY)
}
// Draw faces
path.setFillColor(Color(red: 173/255, green: 216/255, blue: 230/255, opacity: 0.5))
for face in faces {
let points = face.map { projectedVertices[$0] }
path.addLines(points)
path.closeSubpath()
}
// Draw edges
path.setStrokeColor(Color.blue)
path.setLineWidth(2)
for edge in edges {
path.move(to: projectedVertices[edge[0]])
path.addLine(to: projectedVertices[edge[1]])
}
}
}
.onAppear {
Timer.scheduledTimer(withTimeInterval: 0.05, repeats: true) { _ in
self.angleX += 0.02
self.angleY += 0.02
}
}
.background(Color.gray)
}
}
struct ContentView_Previews: PreviewProvider {
static var previews: some View {
ContentView()
}
}
q8
- No flash attention, 4k context
- 9.77 tok/sec, 1813 tokens, 5.58 to first token
- The result didn't compile in Xcode, but I asked the model for the errors and it helped me fix them in another two iterations.
import SwiftUI
struct RotatingCube: View {
@State private var angleX = 0.0
@State private var angleY = 0.0
let vertices: [[Double]] = [
[-1, -1, -1], [1, -1, -1], [1, 1, -1], [-1, 1, -1],
[-1, -1, 1], [1, -1, 1], [1, 1, 1], [-1, 1, 1]
]
let edges: [[Int]] = [
[0, 1], [1, 2], [2, 3], [3, 0],
[4, 5], [5, 6], [6, 7], [7, 4],
[0, 4], [1, 5], [2, 6], [3, 7]
]
let faces: [[Int]] = [
[0, 1, 2, 3], [4, 5, 6, 7], [0, 1, 5, 4],
[2, 3, 7, 6], [1, 2, 6, 5], [0, 3, 7, 4]
]
var body: some View {
GeometryReader { geometry in
let projectedVertices = computeProjectedVertices(geometry.size)
Path { path in
for face in faces {
path.move(to: projectedVertices[face[0]])
for i in 1..<face.count {
path.addLine(to: projectedVertices[face[i]])
}
path.closeSubpath()
}
}
.fill(Color.blue.opacity(0.5))
.overlay(
Path { path in
for edge in edges {
let start = projectedVertices[edge[0]]
let end = projectedVertices[edge[1]]
path.move(to: start)
path.addLine(to: end)
}
}
.stroke(Color.blue, lineWidth: 2)
)
}
.frame(width: 500, height: 500)
.background(Color.gray.opacity(0.1))
.onAppear {
let _ = Timer.scheduledTimer(withTimeInterval: 0.016, repeats: true) { _ in
angleX += 0.01
angleY += 0.01
}
}
}
func computeProjectedVertices(_ size: CGSize) -> [CGPoint] {
let width = size.width
let height = size.height
let halfWidth = width / 2
let halfHeight = height / 2
return vertices.map { v -> CGPoint in
var (x, y, z) = (v[0], v[1], v[2])
let tempY = y * cos(angleX) - z * sin(angleX)
let tempZ = y * sin(angleX) + z * cos(angleX)
y = tempY
z = tempZ
let tempX = x * cos(angleY) - z * sin(angleY)
z = x * sin(angleY) + z * cos(angleY)
x = tempX
let scale = 150 / (z + 4)
let projX = x * scale + halfWidth
let projY = y * scale + halfHeight
return CGPoint(x: projX, y: projY)
}
}
}
struct RotatingCube_Previews: PreviewProvider {
static var previews: some View {
RotatingCube()
}
}
Verdict
The quality of the results (at q8) look very good. In addition, the speed is good enough to use it locally for
development work, particularly for large project changes with many files that require a high context length and would
become very expensive over time on paid models. I'll do more tests soon and report back.