Flat File Data Reader

Resource Page Description
An abstract IDataReader implementation that reads flat files. Enables bulk loading of SQL Server using SqlBulkCopy from flat files with strange formats.

For ordinary delimited or fixed-width files, SSIS is a better tool, but when your file format doesn't work well with these tools or you need a pure .NET solution, consider using this FlatFileDataReader. Includes the option to skip over lines in the file for multi-format files.

Good scenarios are loading from clients (without SSIS), and loading files with multiple record layouts, heders and footers, etc.

Package includes a sample solution.


The FlatFileDataReader base class will process the file line-by-line. Each line will be passed to an abstract class member for you to parse out the column values and store them in local variables.

Here are the abstract members that you would need to implement to read your file:

 
        /**
         * 
         * Implement this to receive each line in the file;
         * After each row is read from the file, this method
         * will be fired.
         * 
         * Return END_OF_FILE if this is the trailer record
         * or to otherwise signal an End-of-file.
         * 
         * Return SKIP if this row is just ignored.
         * 
         * Return READ if this is a valid row
         * 
         * Optionally alter the line (it's passed as a ref).
         * 
         * Throw an exception if the line is invalid.
         * 
         */
        public abstract LineResult ReadLine(ref string line);
 
        /**
         * Readers will call this to get a value
         * Return value should be the proper .NET type for the field         * 
         */
        public abstract object GetValue(int i);
 
        /**
         * These methods return metadata about the fields
         * */
        public abstract Type GetFieldType(int i);
        public abstract string GetName(int i);
        public abstract int FieldCount
        {
            get;
        }

Use the Issue Tracker tab for problems or feature requests, and let me know if you find this sample useful.

David