Monday, January 25, 2010

Reflection on dynamic variables in .NET 4.0

One of the new things in .NET 4.0, are the ability to create dynamic variables, http://msdn.microsoft.com/en-us/library/dd264741(VS.100).aspx.
How does reflection handle this? I decided to go test it.
I started by testing how it handles the dynamic nature of the variable, like this:
dynamic dyn = "string";
Console.WriteLine(dyn.GetType());
dyn = 1;
Console.WriteLine(dyn.GetType());
dyn = 1L;
Console.WriteLine(dyn.GetType());
And happily it turns out we get the correct type after each assignment, the output will look like this:
System.String
System.Int32
System.Int64
The next thing I tested were how dynamic method parameters were handled. I created a method like this:
public void Test(dynamic dyn) { }
What type would reflection return for the dyn parameter?
The following code gets type parameter type of the method, and prints it to the console:
MethodInfo mi = typeof(Program).GetMethod("Test");
ParameterInfo[] pi = mi.GetParameters();
foreach (ParameterInfo info in pi)
   Console.WriteLine(info.ParameterType);
The output will be:
System.Object
Which also are the only possible assumption which can be made about the parameter, as it can be any type deriving from Object.
So how do you see, using reflection, if a variable are declared dynamic? Well at the time of this writing, it does not look like it is possible. This is suspicion are further strengthened by the following paragraph taken from the MSDN article linked in the beginning of the post:
Type dynamic behaves like type object in most circumstances. However, operations that contain expressions of type dynamic are not resolved or type checked by the compiler. The compiler packages together information about the operation, and that information is later used to evaluate the operation at run time. As part of the process, variables of type dynamic are compiled into variables of type object. Therefore, type dynamic exists only at compile time, not at run time.
As all variables of type dynamic are compiled into object types, then reflection returns the correct result by returning Object, when the variable hasn't be assigned another value.

Default Infobright ICE NULL string patch

One thing about Infobright that annoys me a lot, are their exporters default way to indicate NULL values, which is nothing, NULL are indicated by no character. This poses a viewability problem when fields are terminated by for example tabs, then you cannot easily see the null values.

To get around this problem you can use the session variable BH_NULL, and set it to fx. the default MySQL way, \N. But this has to be done for every session/connection.

As I did not want to do this, I created a trivial patch which sets the default NULL string to \N, as in standard MySQL.

The patch can be downloaded below, and has to be applied in the root of the Infobright source files, and works of Infobright 3.3.0.

Patch file:
DefaultNullStr.patch (1 KB)

UPDATE: The patch also works for ICE 3.3.1

Sunday, January 24, 2010

Default Infobright ICE exporter patch

When you export files in the current Infobright version, 3.3.0 and earlier. There are some things you should be aware of. The default exporter are the standard MySQL exporter, but this exporter are not compatible with the default Infobright importer.

For example if you do this, you may run into problems, because of the differences in the exporter and importer:
SELECT * FROM infobrightTable INTO OUTFILE '/tmp/outfile.txt';
LOAD DATA INFILE '/tmp/outfile.txt' INTO TABLE infobrightTable;
Currently there is a bug report about this here. Unfortunately, you'll have to create a user to access the contents.

As described in the bug report, and in other places, you'll have to set the BH_DATAFORMAT session variable to txt_variable, to use the Infobright exporter by default, otherwise you'll may run into format compability problems.

The 'default' exporter will only be used if you are accessing Infobright tables, therefore in my opinion the default exporter should be the Infobright exporter, defined by BH_DATAFORMAT=txt_variable, but at the moment it isn't.

To work around this I've created a very simple patch, which does exactly this. What this patch does is that instead of using the default MySQL exporter, when the BH_DATAFORMAT variable aren't defined, it uses the Infobright exporter instead.

The patch file can be download at the bottom this post, and works for the current version 3.3.0. It has to applied to the file src/storage/brighthouse/core/RCEngine_tools.cpp directly.

Patch file for 3.3.0:
DefaultTxtVariable.patch (1 KB)

UPDATE:
Patch file for 3.3.1:
DefaultTxtVariable331.patch (1 KB)

MySQL and Infobright

At my work place we have used the Infobright Community Edition, http://www.infobright.org/, for a while now, and with great success.
For those who don't know what Infobright is, I recommend that you take a look at this page, What is ICE?, ICE stands for Infobright Community Edition, which is the open source version.
One of Infobrights great strengths are compression, which is top-notch. And for certain analytic queries it performs very, very good.
But there are certainly room for improvements, fx. as Infobright uses its own optimizer, not the default MySQL optimizer, certain queries aren't optimized very well yet, and some aren't supported at all. Unsupported queries will fall back to the standard MySQL optimizer, which doesn't work well with Infobright tables.
Also they are very slow to implement contributed patches, because of this, I've decided to post patches in this blog. Patches which are committed to they're bug tracker, but also patches which are developed specifically for use at my company.

Saturday, January 23, 2010

New MySQL .NET connection string parameter

From MySQL .NET Connector 6.0 a new connection string parameter has been added. The new parameter are called keepAlive, and are used to indicated server ping intervals in seconds.
This parameter are used to ensure database connection aren't dropped, fx. by firewalls due to inactivity. If the connection are dropped, the connection timeout are not obeyed.
A longer discussion and explanation can be found in this bug report, http://bugs.mysql.com/bug.php?id=40684. Currently the parameter aren't documented anywhere in the online .NET Connector documentation.