MultiDataViewCtrl

exampleview.png

Introduction

There are many situation where you only need to be able to show data, and no need for it to be editable. like when viewing logs , traces , or just to show a lot of static text to the user. If the data you want to show is large using CStatic/CEdit controller is not the best choice.

MultiDataViewCtrl is a feature rich custom MFC controller based on CWnd that will show data. And it can show data from files of anysize with a small memory usage with high speed. Even if the file viewed is several gigabytes big the controller will only use small amount of memory.
The data does not need to be located in files. There is a API for adding data directly to the internal memory buffer. So you can have a live trace/log view with a limited scollback buffer, and when the buffer gets full it automaticly removes the top rows.

The data can be shown as ASCII/Unicode/UTF8/Binary or Hex The controller also support color formatting of text, Highlightnig and clickable hyperlinks.

Background

I had already written a controller like this called BFViewCtrl. But it had some problems, One of the biggest problems is that it uses memory filemapping. Memory filemapping is very efficent in most cases. And it works very good. Except when viewing logfiles that continued to grow, when the program that writes the logfils do not open the file in shared-read/write mode, Then the program that writes the logfile could fail to write to the log if the file was being viewed. And that was not good.

There was also some other problem. So instead of fixing the existing controller I desicded to redesign and rewrite it and added a pack of new features like color formatting , highlighing, clickable links, direct memory mode. Only a small chunks of code in BFViewCtrl found a place in MultiDataViewCtrl

Features

  • View files of any size.
  • Live log/trace view mode ( Either with a circular or growing buffer ).
  • Multiple data modes ( ASCII/Unicode/Binary/Hex/UTF8 ).
  • Mouse selection.
  • Smart Selection (Double click on words to select them.)
  • Clipboard support.
  • Save selection to file.
  • File drop support.
  • Font / Color Customizable.
  • Row Lines / Strips or gradient background.
  • Syntax/Color formatting.
  • Word highlighting.
  • Clickable hyperlinks
  • Searching
  • Mouse Wheel zooming
  • Low memory usage.
  • High Speed.
  • Unicode ready.
  • x64 (WIN64) ready.
  • and more....

Todo

  • Printing support.
  • Change Localization Encoding.
  • UnitTests for DataHandler och LineManager classes.

How it works.

When viewing a file that is bigger then 128 kb, the file is read in chunks (unless the option OPTION_ENTIREFILE is used ). The chunks is 64 kb large (change CHUNK_SIZE if you want to have bigger chunks ) and the program is using 2 of them. Actually it is using one 128 kb memory buffer split into 2 parts. It starts by reading the first 64 kb into the buffer and it will parse it and populate an array with LineInfo objects. The LineInfo object contains information on where in the buffer the line start and how long the line is, what formats this line should use and some flags.

Data is parsed as soon as it is loaded or added to the controller. So when the drawing of the text happens, everything is already ready to be rendered. If parsing and formatting should happen in realtime, the rendering of the text would take to much resources and I want drawing to be fast.

When the user then scroll down and are reaching the end of what is in the buffer, the program will read the next 64 kb part. Now the 128 kb buffer is completely full. so when the user continue to scolldown and are reaching the end of the 128kb buffer, the buffer need to be rearranged to make room for the next 64 kb part. It is done by moving the second 64kb part to where the first 64 kb part is located, and overwriting it. But before that is done, all the LineInfo objects for the first part need to be removed. And when the move is complete all the pointers to where lines begin in the LineInfo objects for the block of memory we just moved are now invalid so they need to be realigned. And when all moving and realign and cleanup is done. we can read the next 64 kb part and continue.

Formatting

Text can be color formated (Having multiple fonts and styles on lines is not supported yet. and don't know if it ever will be ). There are 4 levels of formatting. And there are drawn in a special order.

Level 1 - At the bottom we have basic formatting. This can be color syntaxing of words and/or lines, for example if you are viewing a logfile you can format entire rows RED if you are showing an error on that row.

Level 2 - Hyperlinks (if OPTION_HYPERLINK is turned on ofcourse), Then any hyperlink it find when parsing the data will be formated as one and they will be clickable. So when user click on them it will open an browser.

Level 3 - Highlighting. It works like the Highlighting with google toolbar. If you write something in the goolge toolbar and then click on the highlight button all the words you wrote there will be highlighted on the page. All words you add with AddHighlight(...) will be show ontop over the other formatting.

Level 4 - Text Selection is always drawn at the highest level so its always shown so you can select text to copy to clipboard.

HyperLinks

MultiDataViewCtrl can make hyperlinks in the text clickable by adding a special formating on them. Add/remove the option flag OPTION_HYPERLINKS to activate/deactivate it

// Set the option flag for make links clickable
m_DataViewer.SetOption( OPTION_HYPERLINKS );

// Add it to an viewer that is already active.
m_DataViewer.AddOptions( OPTION_HYPERLINKS , true );

Highlighting

To add words that should be highlighted use AddHighlight(...) and specify what colors it should be highlighted with. To activate the highlighting call ApplyHighlights()

m_DataViewer.AddHighlight( "Error" , RGB(255,0,0,) , RGB(0,0,200) );
m_DataViewer.AddHighlight( "Failed" , RGB(255,0,0,) , RGB(0,0,100) );
m_DataViewer.AddHighlight( "Success" , RGB(0,255,0,) , RGB(50,50,50) );

m_DataViewer.ApplyHighlighs();

Custom Formatting

To controll how the basic formatting is done you need to tell the formatter on how you want to format it. Inlucde are a GenericDocumentFormater. And if that is not good enouth for you. You can write your own custom colorformater.
To Create your own formatter you create a class and inherit that from IDocumentFormater and then set it to CMDataViewCtrl with SetDocumentFormater( ... ) then your class will be called during the parsing process so it can set formatting information. You can also use the included GenericDocumentFormater to add your own formatting.

CGenericDocumentFormater* pFormater = (CGenericDocumentFormater*)m_DataViewer.GetDocumentFormater();
 if( pFormater == NULL )
 {
   m_DataViewer.SetDocumentFormater( (IDocumentFormater*)-1 , true ); // -1 will create the CGenericDocumentFormater.
   pFormater = (CGenericDocumentFormater*)m_DataViewer.GetDocumentFormater();
 }

 if( pFormater )
 {
   // format the word 'warning' to blue
   pFormater->AddFormat( "warning" , FMTHLP_WORD , FMT_STYLE_COLOR|FMT_DOCUMENT , RGB(0,0,255) );

   // if the substring 'fatal' is found format the entire line as red
   pFormater->AddFormat( "fatal" , FMTHLP_SUBSTRING|FMTHLP_LINE , FMT_STYLE_COLOR|FMT_DOCUMENT , RGB(255,0,0) );
 }


 // Need to be run if applying to an existing document. 
 // If formats are set before document is loaded. Then this is not needed
 m_DataViewer.ApplyDocumentFormats();

Direct Memory Mode (Live log/trace view mode)

Instead of viewing data from a file. The controller can be open in Direct memory mode and data can be added directly to the view buffer with AddData(...) / AddText(...) This is usefull if using MultiDataViewCtrl as a live log/trace view.

AddData(...) is thread-safe so it is safe to use it across thread. If AddData(..) is called from another thread the data is added to a queue that is then picking it up and added to the controller from the GUI thread. But this way of adding data have one side effect. If data comes in from the workerthread and then from GUI thread the data from the GUI thread might be shown before the workthread data in the viewer. this is because data from the GUI thread is added directly and not via the queue.

This can be fixed by settings SetGarantiedDataOrder() to true. Then data is always added via the queue even for AddData/AddText calls comming from the GUI thread. So why not always do that you might ask ? Adding data via the queue is a little slower and will consume a little more memory. normaly this is not a problem. but in some situation you do not want this extra overhead.

// Open the viewer in DirectMemory mode.
m_DataViewer.OpenData( NULL , DATAHANDLER_ASCII , 0 , OPEN_DIRECT_MEMORY );
m_DataViewer.SetGarantiedDataOrder(true);

// From Any thread this can be used
m_pDataViewer->AddText( "User 'Freddy Cruger' logged in" );

As default is will use a cirular buffer just like when viewing files. When the buffer is full the first chunk part is removed and overwritten by the second part, But if you added the flag OPEN_DATA_GROW, when opening, the buffer is now grown when needed and nothing is lost. But be carefull, If the programa that adds data to the controller is very talky the buffer will grow and grow and grow until you run out of memory.

if the flag OPTION_STICKTOBOTTOM is use then the controller will auto scroll to the bottom of the text when data is added to it if the current position is already at the end.

m_DataViewer.AddOptions( OPTION_STICKTOBOTTOM );
 m_DataViewer.SetBufferSize( 1024 * 1024 ); // 1MB buffer
 m_DataViewer.OpenData( NULL , DATAHANDLER_ASCII , 0 , OPEN_DIRECT_MEMORY );

Limitations

One of the limitation is the clipboard. Since it is possible to view file of any size I had to limit the copy to clipboard feature. It is limited to 10MB and can be changed if needed be changed MAX_CLIPBOARD_SIZE.

Another limitation is formatting of text. Formatting is reset at the end of the line, This is done on purpose since we can view files of any size so data is read in chunk we can not know if the data that are viewed should have had some formatting that should have been set before in another chunk. Allowing Formatting to 'bleed' over to the next line would have resulted in some very strange behavior and would be a mess to sort out (I think).

The source code

MultiDataViewCtrl contains around 32 .cpp and .h files. So the easiest way to add it to your project is to create a subfolder in your solution and then do add existing files to it and add the entire MultiDataViewCtrl/* folder. And you should only need to access CMDataViewCtrl so it should be enougth to include just that in your files where you need to access CMDataViewCtrl.

The code is using STL and I use a couple of defines for STL that I use a lot in my projects so you need to include zSTLDef.h in stdafx.h

// in stdafx.h
#include "MultiDataViewCtrl/zSTLDef.h"

// In the classes that using the MultiDataViewCtrl
#include "MultiDataViewCtrl/MDataViewCtrl.h"
DataHandler.cpp/.h              - Base class for all text handlers
 |-- AsciiHandler.cpp/.h        - Renderer and Handler for AscII 
      |-- UnicodeHandler.cpp/.h - Renderer and Handler for Unicode  	
      |-- BinaryHandler.cpp/.h  - Renderer and Handler for Binary
      |    |- HexHandler.cpp/.h - Renderer and Handler for Hex
      |-- UTF8Handler.cpp/.h    - Renderer and Handler for UTF8

DataReader.cpp/.h	   - Handling the Reading and main buffer 
LineManager.cpp/.h         - Handling the Parsed lines 
                           - for the data in the buffer

MDataViewCtrl.cpp/.h       - The CWnd Derived controller.
TextSchema.cpp/.h          - Handing the colors of fonts to use.

ISearch.h                      - Interface for search API
 |-- DefaultDataSearch.cpp/.h  - Default search impementation for Ascii/Unicode

IDocumentFormater.h                - Interface for DocumentFormater
 |-- GenericDocumentFormter.cpp/.h - Default DocumentFormater
FormatHelper.cpp/.h                - Helper class for GenericDocumentFormater

HighLightManager.h         - Handles the Highlighting,

Gradient.cpp/.h            - Used for drawing gradient background
MemDC.h                    - Used for drawing to memory dc

Most of the head scratching code is found in DataHandler.cpp and LineManager.cpp, If bugs are found, they are most likley going to be in any of those two files.

There are plans to create some unit-tests for them (Yes I know. I should have started with tests from the beginning) because that are needed so the functionality is guaranteed when changing the code. Because any changes to that code require testing to ensure that everything still works.

Create the controller (For dialog)

Adding CMDataViewCtrl to a dialog.

properties.jpg

  1. Add a custom controller to the layout where you want to have it.
  2. Set the ID of the properties to something like IDC_DATAVIEW.
  3. Class name in properties must be "MultiDataViewCtrl".
  4. Then set the style flags you want. like 0x56810000. if you do not want any border (WS_BORDER) remove 0x00800000 from the value.
  5. Then Add a CMDataViewCtrl variable to you dialog class
    CMDataViewCtrl  m_DataViewer;
  6. To connect the custom controller in the dialog layout with the m_DataViewer variable you need to add some code into DoDataExchange.
    void CMultiViewCtrlDemo::DoDataExchange(CDataExchange* pDX)
    {
    	CDialog::DoDataExchange(pDX);
    	DDX_Control(pDX, IDC_DATAVIEW , m_DataViewer );
    }
  7. Then configure the controller how you want it in OnInitDialog().

Create the controller (Dynamically)

You can also create the controller dynamicly with

CRect rc(5,5,400,400);
int nID = 1020;
CMDataViewCtrl *pViewer = new CMDataViewCtrl();
pViewer->Create( this , rc , nID , WS_VISIBLE | WS_BORDER | WS_CHILD );

Configuring

There are a lot of options to customize the look and feel of the viewer. Some of the most importent options are the following.

Enable drop file support

 m_DataViewer.DragAcceptFiles( TRUE );

what mode the dropped files should be open with.

m_DataViewer.SetDropOpenOptions( OPEN_ENTIREFILE ); 

SetOptions

 #define OPTION_HYPERLINKS      0x00000100L // Allow for Hyperlink formatting 
 #define OPTION_LINES           0x00000200L // Line under every row 
 #define OPTION_LINESTRIPS      0x00000800L // Every even line is drawn with one color,
                                            // odd lines drawn with another.
 #define OPTION_SELECTION       0x00001000L // Draw selection. 
 #define OPTION_CLIPBOARD       0x00002000L // Allow copy to clipboard 
 #define OPTION_SMARTSELECTION  0x00004000L // Allow smart selection 
 #define OPTION_DBLCLK_GOLINK   0x00008000L // GoTo Hyper link on double click 
 #define OPTION_CLK_GOLINK      0x00010000L // GoTo Hyper link on single click 
 #define OPTION_STICKTOBOTTOM   0x00020000L // If reload or new data is added. 
                                            // keep vertical scroll at the bottom. 
 #define OPTION_BK_GRADIENT     0x00100000L // Draw Gradient color as background. 
 #define OPTION_BK_IMAGE        0x00200000L // Draw an background image 
                                            // (Not fully implemented yet)
 #define OPTION_WHEELZOOM       0x00400000L // Mouse Wheel Zoom
                                            // /Inc/dec font size with ctrl+mouse wheel)
				
 m_DataViewer.SetOptions(OPTION_HYPERLINKS|OPTION_CLIPBOARD|OPTION_STICKTOBOTTOM, Refresh );

Set Font and Colors

The class CTextSchema store all color and font configuration. So either create a new CTextSchema and set that or get the current one and modify that.

 CTextSchema* pTextSchema = m_wndDataView.GetSchema();
 pTextSchema.SetColorLink( RGB(255, 0, 0), RGB(0, 255, 255));
 // or 
 pTextSchema->SetColor(COLOR_LINK_TEXT, RGB(255, 0, 0);
 pTextSchema->SetColor(COLOR_LINK_BK, RGB(0, 255, 255));

example_styles.png

Open file

To Open a file to view you call will need to call OpenData.

OpenData( const TCHAR* strFilename , int nMode = DATAHANDLER_AUTO, 
           int nDataIdentSize = 0 , short nOpenOption = 0 );

In nMode specify how you want to open and show the data as.

#define DATAHANDLER_AUTO        0 // Try to autodetect format
#define DATAHANDLER_ASCII       1 // Open the data as ASCII
#define DATAHANDLER_UNICODE     2 // Open the data as Unicode
#define DATAHANDLER_UNICODE_BE  3 // Open the data as Unicode BigEndian
                                  // (support not implemented yet)
#define DATAHANDLER_UTF8        4 // Open the data as UTF8
#define DATAHANDLER_BINARY      5 // Open the data as Binary
#define DATAHANDLER_HEX         6 // Open the data as Hex

nDataIdentSize is the number of byte to skip from the beginning of the file. Unicode and UTF8 file often have a bytemarker in the begining of the file that is there to identify the file format and we do not want to view that so then set nDataIdentSize to 2 for unicode.

nOpenOption are some optional flag on how data should be handled.

// Always read in the entire file into memory.
#define OPEN_ENTIREFILE     0x01  

// Use DirectMemory mode when adding data to view.
// The controller will NOT load the data from a file. 
// It will be given data by the AddData(...)/AddText(...) functions. 
// They functions will add data direct to the memory buffer
// (eg used often when using the controller for trace/live log viewer.)
#define OPEN_DIRECT_MEMORY  0x02  

// The buffer will grow instead of being fixed size and circular.
// Is only valid if also OPEN_DIRECT_MEMORY is specified,
// But be careful it can eat up your entire memory.
#define OPEN_DATA_GROW     0x04

Example of customizing the style

 // Set the font we want to use
 m_wndDataView.SetFont( _T("Courier New") , 9 );

 
 // Get the textschema so we can modify text and background colors.
 CTextSchema* pSchema = m_wndDataView.GetSchema();
 
 // Set Default text color
 pSchema->SetColorDefault( RGB(0,100,100), RGB(100,100,100) );
 
 // Set Gradient mode for the background
 pScheme->SetGradientMode(GRADIENT_HORIZONTAL);
		 
 // Get the document formater so we can tell it to format our text the way we want it.
 CGenericDocumentFormater* pFormater = (CGenericDocumentFormater*)m_DataViewer.GetDocumentFormater();
 if( pFormater == NULL )
 {
   // -1 will create default CGenericDocumentFormater.
   m_DataViewer.SetDocumentFormater( (IDocumentFormater*)-1 , true ); 
   pFormater = (CGenericDocumentFormater*)m_DataViewer.GetDocumentFormater();
 }

 if( pFormater )
 {
   // if the substring 'error' is found format the entire line as red
   pFormater->AddFormat( "OK" , FMTHLP_SUBSTRING|FMTHLP_LINE , FMT_STYLE_COLOR|FMT_DOCUMENT , -1 , RGB(0,240,0) );
   pFormater->AddFormat( "Failed" , FMTHLP_SUBSTRING|FMTHLP_LINE , FMT_STYLE_COLOR|FMT_DOCUMENT , -1, RGB(220,0,0) );
 }	
 
 // Add Highlighting for the words OK and Failed
 m_DataViewer.AddHighlight( _T("user1") , -1 , RGB(255,255,32); // Highlight all "user1" substring it finds
 m_DataViewer.AddHighlight( _T("user6") , -1 , RGB(0,255,255)); // Highlight all "user6" substring it finds

formatingandhighlighting.png

Notifications

There are a few notification messages that are used to notify the owner window when things have happend.

 #define MDVNM_FILE_OPENED       1		// A file have been loaded into the viewer
 #define MDVNM_DYNDATA_OPENED    2		// A Dynamic Data viewer has opened
 #define MDVNM_FILE_RELOADED     3		// A file have been reloaded
 #define MDVNM_SELECTIONCHANGED  4		// Selection have Change
 #define MDVNM_DATA_ADDED        5		// Data has been added to the viewer
                                                //( Only when it is in dynamic mode )
 #define MDVNM_BUFFER_CHANGED    6		// the buffer has changed.
 #define MDVNM_VIEWMODE_CHANGED  7		// ViewMode is changed

To catch the notification message you need can do this in you dialog.

 // in .h file
 afx_msg void OnMDVFileOpen(NMHDR *pNotifyStruct, LRESULT* pResult) ;

 // in .cpp file
 BEGIN_MESSAGE_MAP(CMYDlg, CDialog)
	ON_NOTIFY( MDVNM_FILE_OPENED , IDC_DATAVIEW , OnMDVFileOpen ) 
 END_MESSAGE_MAP()

 void CMYDlg::OnMDVFileOpen(NMHDR* pNotifyStruct, LRESULT* pResult) 
 {
	...
	...
 }

History

For full history check the header in MDataViewCtrl.cpp

  • v1.4 - 2008-10-10 - First Public release.
  • v1.3 - 2008-09-20 - Personal version, Never released.
  • v1.2 - 2008-04-20 - Personal version, Never released.
  • v1.1 - 2007-11-26 - Personal version, Never released.
  • v1.0 - 2006-04-21 - Personal version, Never released.

Credits

  • MemDC - Keith Rule
  • Modified version of CGradient by Darkoman

 

Tags