In this post I will show how to read/write different data type values to and from a byte buffer.
This should prove useful when working with block data or pages.
In this scenario, pages describes database pages or similar block data mechanisms.
Top
In this post...
Top
How To Handle Typed Values
The aim is to read/write different typed values from a byte array.
In C++ one can achieve this by using a union.
In C# we achieve this by using a struct/class using explicit layout.
Explicit layout is the key here as it ensures types are aligned correctly.
The basic idea is to have a number of byte values offset from zero to maximum bytes required.
Then add basic types, such as int 16, int 32, int 64, float, etc all at offset zero.
Then one can set a float value and read of the four byte values.
The same can be done for most basic types, set the value and the read the corresponding byte values.
To read typed values, simply populate the byte values accordingly then read the basic type info (int 16, int 32, int float, etc).
For those with a COM background this approach is similar to the VARIANT type.
Top
The Union Value Type
using System;
using System.Runtime.InteropServices;
namespace Blog
{
[StructLayout(LayoutKind.Explicit)]
public struct Value
{
[FieldOffset(0)] public byte B0;
[FieldOffset(1)] public byte B1;
[FieldOffset(2)] public byte B2;
[FieldOffset(3)] public byte B3;
[FieldOffset(4)] public byte B4;
[FieldOffset(5)] public byte B5;
[FieldOffset(6)] public byte B6;
[FieldOffset(7)] public byte B7;
[FieldOffset(0)] public float Float;
[FieldOffset(0)] public double Double;
[FieldOffset(0)] public Int16 Int16;
[FieldOffset(0)] public Int32 Int32;
[FieldOffset(0)] public Int64 Int64;
}
}
The above struct declaration shows eight bytes, B0-B7 for data transfer.
If using the decimal type this will need to be expanded to include 16 bytes!
As can be seen, B0-B7 is byte aligned and offset accordingly.
Note also, that basic types float, double, Int16, etc all start at offset zero.
So, if one sets the Float field, the corresponding 4 bytes can be read from B0-B3.
To reverse the action, set fields B0-B3, then read the float value.
It is imperative that one knows how many bytes are required for a typed value.
The above structure should cover most data types bar decimal.
DateTime values can also be stored by first converting to long and using the Int64 field or bytes B0-B7 to recreate.
Top
Testing The Union Value Type
The above can be tested using the following code...
static void RawTest()
{
Value value = new Value();
byte[] data = new byte[128];
int cursor = 0;
Int16 v1 = Int16.MaxValue - 1067;
Int32 v2 = Int32.MaxValue - 10067;
float v3 = float.MaxValue - 1.07896f;
value.Int16 = v1;
data[cursor++] = value.B0;
data[cursor++] = value.B1;
value.Int32 = v2;
data[cursor++] = value.B0;
data[cursor++] = value.B1;
data[cursor++] = value.B2;
data[cursor++] = value.B3;
value.Float = v3;
data[cursor++] = value.B0;
data[cursor++] = value.B1;
data[cursor++] = value.B2;
data[cursor++] = value.B3;
cursor = 0;
value.B0 = data[cursor++];
value.B1 = data[cursor++];
var v1Result = value.Int16;
value.B0 = data[cursor++];
value.B1 = data[cursor++];
value.B2 = data[cursor++];
value.B3 = data[cursor++];
var v2Result = value.Int32;
value.B0 = data[cursor++];
value.B1 = data[cursor++];
value.B2 = data[cursor++];
value.B3 = data[cursor++];
var v3Result = value.Float;
Console.WriteLine(v1);
Console.WriteLine(v1Result);
Console.WriteLine(v2);
Console.WriteLine(v2Result);
Console.WriteLine(v3);
Console.WriteLine(v3Result);
}
Top
Writing Values To A Byte Array
Figure 2 illustrates a byte array filled with values from the previous test code.
As the diagram illustrates, data values are byte aligned accordingly.
That is, a 16 bit-integer requires 2 bytes, a float or 32-bit integer requires 4 bytes.
This approach works well for data blocks comprising of byte arrays.
Simply create a byte array, write values and save to disk.
Conversely, load a byte array block from disk into memory, then proceed to read actual values.
Top
Improving The Value Interface
The current method of reading/writing values is somewhat verbose.
One needs to track the byte array offset (cursor) and the offset to add following a read or write.
In addition, explicitly reading and writing to the Value's byte fields (B0...BN) is tiresome and error-prone.
The following should help alleviate these problems...
- Specify the byte array and initial offset.
- Specify a cursor that is relative to the specified initial offset.
- Update the cursor accordingly following a read or write operation.
- Allow the cursor's position to be set manually, the cursor will always be relative to the specified offset.
The following class, ByteBuffer, implements the above features.
using System;
namespace Blog
{
public class ByteBuffer
{
private Value _value = new Value();
private int _internalCursor;
private int _offset;
private readonly byte[] _buffer;
public int Cursor => _internalCursor - _offset;
public ByteBuffer(byte[] buffer, int offset)
{
_buffer = buffer;
_offset = offset;
_internalCursor = offset;
}
public ByteBuffer SetCursor(int position)
{
_internalCursor = _offset + position;
return this;
}
public ByteBuffer Int16(Int16 value)
{
_value.Int16 = value;
_buffer[_internalCursor++] = _value.B0;
_buffer[_internalCursor++] = _value.B1;
return this;
}
public ByteBuffer Int16(out Int16 result)
{
_value.B0 = _buffer[_internalCursor++];
_value.B1 = _buffer[_internalCursor++];
result = _value.Int16;
return this;
}
public ByteBuffer Int32(Int32 value)
{
_value.Int32 = value;
_buffer[_internalCursor++] = _value.B0;
_buffer[_internalCursor++] = _value.B1;
_buffer[_internalCursor++] = _value.B2;
_buffer[_internalCursor++] = _value.B3;
return this;
}
public ByteBuffer Int32(out Int32 result)
{
_value.B0 = _buffer[_internalCursor++];
_value.B1 = _buffer[_internalCursor++];
_value.B2 = _buffer[_internalCursor++];
_value.B3 = _buffer[_internalCursor++];
result = _value.Int32;
return this;
}
public ByteBuffer Float(float value)
{
_value.Float = value;
_buffer[_internalCursor++] = _value.B0;
_buffer[_internalCursor++] = _value.B1;
_buffer[_internalCursor++] = _value.B2;
_buffer[_internalCursor++] = _value.B3;
return this;
}
public ByteBuffer Float(out float result)
{
_value.B0 = _buffer[_internalCursor++];
_value.B1 = _buffer[_internalCursor++];
_value.B2 = _buffer[_internalCursor++];
_value.B3 = _buffer[_internalCursor++];
result = _value.Float;
return this;
}
}
}
Top
Using The ByteBuffer Class
Using the ByteBuffer class is fairly straightforward.
Simply call the constructor with a byte array and offset.
No data can be read or written before the offset.
A sample test/driver program now follows...
static void Main(string[] args)
{
byte[] data = new byte[128];
ByteBuffer buffer = new ByteBuffer(data, 5);
Int16 v1 = Int16.MaxValue - 1067;
Int32 v2 = Int32.MaxValue - 10067;
float v3 = float.MaxValue - 1.07896f;
int bytesWritten = buffer
.Int16(v1)
.Int32(v2)
.Float(v3)
.Cursor;
Console.WriteLine($"{bytesWritten} bytes written to buffer.");
int bytesRead = buffer
.SetCursor(0)
.Int16(out var v1Read)
.Int32(out var v2Read)
.Float(out var v3Read)
.Cursor;
Console.WriteLine($"{bytesRead} bytes read from buffer.");
Console.WriteLine($"v1Write:{v1} - v1Read:{v1Read}");
Console.WriteLine($"v2Write:{v2} - v2Read:{v2Read}");
Console.WriteLine($"v3Write:{v3} - v3Read:{v3Read}");
}
The above code wites the following to the console.
Notice how the test program uses an offset of 5 when constructing the ByteBuffer.
The correct bytes read/wrriten of ten is still returned.
The first five bytes in this example will be zeroed.
Top
