Thread Computation Examples
Block 0 - Thread 2 (Boundary)
Window: [3, 4, 5] (uses overlap)
Result: 3×0.33 + 4×0.33 + 5×0.33 = 4.0
Block 1 - Thread 3 (Boundary)
Window: [8, 9, 10] (uses overlap)
Result: 8×0.33 + 9×0.33 + 10×0.33 = 9.0
Block 2 - Thread 2 (Edge)
Window: [11, 12, 0] (zero padding)
Result: 11×0.33 + 12×0.33 + 0×0.33 = 7.67
Block 2 - Thread 3 (Edge)
Window: [12, 0, 0] (zero padding)
Result: 12×0.33 + 0×0.33 + 0×0.33 = 4.0

1D Convolution with Block Boundary Handling

Setup & Global Input Array
TPB: 4  |  CONV_2: 3  |  SIZE_2: 12
Indices:
0
1
2
3
4
5
6
7
8
9
10
11
Values:
1
2
3
4
5
6
7
8
9
10
11
12
Block 0: 0-3  |  Block 1: 4-7  |  Block 2: 8-11
Kernel: [0.33, 0.33, 0.33]
Function Signature
fn conv_1d_block_boundary[ in_layout: Layout, out_layout: Layout, conv_layout: Layout, dtype: DType ]( output: LayoutTensor[mut=False, dtype, out_layout], a: LayoutTensor[mut=False, dtype, in_layout], b: LayoutTensor[mut=False, dtype, conv_layout], ):
Block 0: Shared Memory Loading
Step 1: Initialize Variables and Allocate Extended Shared Memory
// Get thread indices global_i = block_dim.x * block_idx.x + thread_idx.x local_i = thread_idx.x // Allocate extended shared memory for overlap shared_a = tb[dtype]().row_major[TPB + CONV_2 - 1]().shared().alloc() shared_b = tb[dtype]().row_major[CONV_2]().shared().alloc() // Size calculation: 4 + 3 - 1 = 6 elements
Step 2: Load Main Data (Each Thread Loads One Element)
// Each thread loads its corresponding input element if global_i < SIZE_2: shared_a[local_i] = a[global_i] else: shared_a[local_i] = 0 // Zero padding for out-of-bounds // Block 0 threads load: // Thread 0: shared_a[0] = a[0] = 1 // Thread 1: shared_a[1] = a[1] = 2 // Thread 2: shared_a[2] = a[2] = 3 // Thread 3: shared_a[3] = a[3] = 4
Shared Memory Layout After Main Loading:
1
2
3
4
?
?
■ Main Data Loaded    ■ Overlap (to be loaded)
Step 3: Load Overlap Data (Boundary Threads Only)
// Only first (CONV_2 - 1) threads load overlap data if local_i < CONV_2 - 1: // Threads 0 and 1 // Calculate indices from next block next_idx = global_i + TPB if next_idx < SIZE_2: shared_a[TPB + local_i] = a[next_idx] else: // Initialize out-of-bounds elements to 0 shared_a[TPB + local_i] = 0 // Block 0 overlap loading: // Thread 0: next_idx = 0 + 4 = 4, shared_a[4] = a[4] = 5 // Thread 1: next_idx = 1 + 4 = 5, shared_a[5] = a[5] = 6
1
2
3
4
5
6
Complete Shared Memory for Block 0
Step 4: Load Kernel Data
// Load kernel elements into shared memory if local_i < CONV_2: shared_b[local_i] = b[local_i] // Synchronize all threads before computation barrier()
Block 0: Convolution Computation
Convolution Algorithm
// Perform convolution for each thread's output element if global_i < SIZE_2: var local_sum: output.element_type = 0 // Parameterized loop for compile-time unrolling @parameter for j in range(CONV_2): if global_i + j < SIZE_2: local_sum += shared_a[local_i + j] * shared_b[j] output[global_i] = local_sum
Thread 0 (global_i=0, local_i=0)
Window: shared_a[0:3] = [1, 2, 3]
Computation:
1×0.33 + 2×0.33 + 3×0.33
Result: 2.0
Thread 1 (global_i=1, local_i=1)
Window: shared_a[1:4] = [2, 3, 4]
Computation:
2×0.33 + 3×0.33 + 4×0.33
Result: 3.0
Thread 2 (global_i=2, local_i=2) *
Window: shared_a[2:5] = [3, 4, 5]
Computation:
3×0.33 + 4×0.33 + 5×0.33
Result: 4.0
Uses overlap data!
Thread 3 (global_i=3, local_i=3) *
Window: shared_a[3:6] = [4, 5, 6]
Computation:
4×0.33 + 5×0.33 + 6×0.33
Result: 5.0
Uses overlap data!
Block 1: Shared Memory Loading
Block 1 Thread Mapping
// Block 1 thread indices and global positions block_idx.x = 1 block_dim.x = 4 // Thread mapping: // Thread 0: global_i = 1*4 + 0 = 4, local_i = 0 // Thread 1: global_i = 1*4 + 1 = 5, local_i = 1 // Thread 2: global_i = 1*4 + 2 = 6, local_i = 2 // Thread 3: global_i = 1*4 + 3 = 7, local_i = 3
Step 1: Load Main Data (Indices 4-7)
// Each thread in Block 1 loads its element if global_i < SIZE_2: shared_a[local_i] = a[global_i] // Block 1 main data loading: // Thread 0: shared_a[0] = a[4] = 5 // Thread 1: shared_a[1] = a[5] = 6 // Thread 2: shared_a[2] = a[6] = 7 // Thread 3: shared_a[3] = a[7] = 8
Shared Memory Layout for Block 1:
5
6
7
8
?
?
Step 2: Load Overlap Data from Block 2 (Indices 8-9)
// Boundary threads load overlap from Block 2 if local_i < CONV_2 - 1: // Threads 0, 1 next_idx = global_i + TPB if next_idx < SIZE_2: shared_a[TPB + local_i] = a[next_idx] else: shared_a[TPB + local_i] = 0 // Block 1 overlap loading: // Thread 0: next_idx = 4 + 4 = 8, shared_a[4] = a[8] = 9 // Thread 1: next_idx = 5 + 4 = 9, shared_a[5] = a[9] = 10
5
6
7
8
9
10
Complete Shared Memory for Block 1
Block 1: Convolution Computation
Global Memory Access by Block 1:
1
2
3
4
5
6
7
8
9
10
11
12
Thread 0 (global_i=4, local_i=0)
Window: shared_a[0:3] = [5, 6, 7]
Computation:
5×0.33 + 6×0.33 + 7×0.33
Result: 6.0
Thread 1 (global_i=5, local_i=1)
Window: shared_a[1:4] = [6, 7, 8]
Computation:
6×0.33 + 7×0.33 + 8×0.33
Result: 7.0
Thread 2 (global_i=6, local_i=2) *
Window: shared_a[2:5] = [7, 8, 9]
Computation:
7×0.33 + 8×0.33 + 9×0.33
Result: 8.0
Uses overlap data!
Thread 3 (global_i=7, local_i=3) *
Window: shared_a[3:6] = [8, 9, 10]
Computation:
8×0.33 + 9×0.33 + 10×0.33
Result: 9.0
Uses overlap data!
Block 2: Shared Memory Loading (Edge Case)
Block 2 Thread Mapping
// Block 2 thread indices and global positions block_idx.x = 2 block_dim.x = 4 // Thread mapping: // Thread 0: global_i = 2*4 + 0 = 8, local_i = 0 // Thread 1: global_i = 2*4 + 1 = 9, local_i = 1 // Thread 2: global_i = 2*4 + 2 = 10, local_i = 2 // Thread 3: global_i = 2*4 + 3 = 11, local_i = 3
Step 1: Load Main Data (Indices 8-11)
// Each thread in Block 2 loads its element if global_i < SIZE_2: shared_a[local_i] = a[global_i] // Block 2 main data loading: // Thread 0: shared_a[0] = a[8] = 9 // Thread 1: shared_a[1] = a[9] = 10 // Thread 2: shared_a[2] = a[10] = 11 // Thread 3: shared_a[3] = a[11] = 12
Shared Memory Layout for Block 2:
9
10
11
12
?
?
Step 2: Handle Out-of-Bounds Overlap (Zero Padding)
// Block 2 tries to load indices 12, 13 (out of bounds!) if local_i < CONV_2 - 1: // Threads 0, 1 next_idx = global_i + TPB if next_idx < SIZE_2: shared_a[TPB + local_i] = a[next_idx] else: shared_a[TPB + local_i] = 0 // Zero padding! // Block 2 overlap attempt: // Thread 0: next_idx = 8 + 4 = 12 >= 12 (SIZE_2) → shared_a[4] = 0 // Thread 1: next_idx = 9 + 4 = 13 >= 12 (SIZE_2) → shared_a[5] = 0
9
10
11
12
0
0
Complete Shared Memory for Block 2 (with zero padding)
⚡ Block 2: Convolution Computation
Global Memory Access by Block 2:
1
2
3
4
5
6
7
8
9
10
11
12
⚠️ No overlap data available - using zero padding
Thread 0 (global_i=8, local_i=0)
Window: shared_a[0:3] = [9, 10, 11]
Computation:
9×0.33 + 10×0.33 + 11×0.33
Result: 10.0
Thread 1 (global_i=9, local_i=1)
Window: shared_a[1:4] = [10, 11, 12]
Computation:
10×0.33 + 11×0.33 + 12×0.33
Result: 11.0
Thread 2 (global_i=10, local_i=2) ⚠️
Window: shared_a[2:5] = [11, 12, 0]
Computation:
11×0.33 + 12×0.33 + 0×0.33
Result: 7.67
Uses zero padding!
Thread 3 (global_i=11, local_i=3) ⚠️
Window: shared_a[3:6] = [12, 0, 0]
Computation:
12×0.33 + 0×0.33 + 0×0.33
Result: 4.0
Uses zero padding!
📊 Complete Convolution Results
Input Array:
1
2
3
4
5
6
7
8
9
10
11
12
↓ CONVOLUTION ↓
Output Array:
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
10.0
11.0
7.67
4.0
■ Block 0 Results    ■ Block 1 Results    ■ Block 2 Results
Complete Implementation Summary
fn conv_1d_block_boundary[ in_layout: Layout, out_layout: Layout, conv_layout: Layout, dtype: DType ]( output: LayoutTensor[mut=False, dtype, out_layout], a: LayoutTensor[mut=False, dtype, in_layout], b: LayoutTensor[mut=False, dtype, conv_layout], ): // 1. Calculate thread indices global_i = block_dim.x * block_idx.x + thread_idx.x local_i = thread_idx.x // 2. Allocate extended shared memory for overlap shared_a = tb[dtype]().row_major[TPB + CONV_2 - 1]().shared().alloc() shared_b = tb[dtype]().row_major[CONV_2]().shared().alloc() // 3. Load main data if global_i < SIZE_2: shared_a[local_i] = a[global_i] else: shared_a[local_i] = 0 // 4. Load overlap data (boundary handling) if local_i < CONV_2 - 1: next_idx = global_i + TPB if next_idx < SIZE_2: shared_a[TPB + local_i] = a[next_idx] else: shared_a[TPB + local_i] = 0 // Zero padding // 5. Load kernel data if local_i < CONV_2: shared_b[local_i] = b[local_i] // 6. Synchronize all threads barrier() // 7. Perform convolution if global_i < SIZE_2: var local_sum: output.element_type = 0 @parameter for j in range(CONV_2): if global_i + j < SIZE_2: local_sum += shared_a[local_i + j] * shared_b[j] output[global_i] = local_sum
Key Design Principles
Performance Benefits
• Memory Coalescing: Sequential access patterns
• Shared Memory: Fast on-chip storage
• Bandwidth Efficiency: Each element loaded once
• Parallel Execution: All threads work simultaneously
Critical Considerations
• Boundary Safety: Zero padding prevents crashes
• Edge Effects: Last block has artifacts
• Memory Size: TPB + CONV_2 - 1 allocation
• Synchronization: barrier() before computation