Section 7: Leveraging the Run-Time Data Cache

NetCharts Server provides API access to the run-time data cache. This is an extremely useful feature, for a number of important reasons:

  • Developers with remote application servers can remotely store data into the cache by passing in XML data, JDBC Result Sets, or Object arrays.
  • Cached data can be filtered, sorted, transposed, aggregated, and auto-formatted into chart parameters, in exactly the same way as Named Data Set (NDS) data.
  • Developers can extract data from any source, perform business logic manipulation on the data
    and then store it into the cache for display.

In this section we will demonstrate how to set up the data sources, charts, and tables for cache use and then various processing and integration options.

NOTE: This example is intended to show the application of business logic to a particular data set, so it is geared to a specific problem. The premise of this problem is that we have raw data that represents search engine impressions and click thru rates for a variety of keywords. We want our charts and tables to show click thru rates for all words, but highlight those keywords that exceed a given clickthru rate threshold.

Step 1 – Create a Project

Creating a Project

Open a browser window, and go to the NetCharts Server Developer Console. The URL, if you are on the same machine as the NetCharts Server installation, is:

http://localhost:8001/Developer/index.jsp

If the NetCharts Server is on another machine or port, use the appropriate server name and port number for that remote machine.

The Developer Console’s main page displays the Project List, which represents a list of all the projects available on this machine.

Click the NEW button above the Project List table.

Enter in the new project name, call it Cache Project and click the CREATE button.

A new, empty project is created. The browser is now showing the Project Files panel.

Step 2 – Create a Data Source

For this example, we will use a cache data source that we’ll configure with default data.

Create a Cache NDS

  • Click the NEW button above the Project Files panel.
  • Enter the name cachedata in the name field.
  • Select Data/NDS from the Type menu.
  • Click the CREATE button. The Named Data Set Design page is shown.
  • Select Cache from the Type menu.
  • Enter the sample data shown here into the Default Data text area:
     

    Keyword,Clickthru,Color
    Apples,2,green
    Oranges,3,green
    Pears,5,purple
    Peaches,4,green

     
    Make sure there are no extra spaces between the items. The default data helps us design our charts without needing the real data available.

  • Set the data cache Key to AdCache. Set the Lifespan to 3.
  • Click the TEST button. Our sample data appears as shown here:
     
    dev
     
  • Close the test window and then click DONE.

Create the Raw Data

We will create an NDS that will give us a raw data source. In practice, the developer can use any available data source – the HSQL NDS source used here is just to simplify the example.

  • Click the NEW button above the Project Files panel.
  • Enter the name rawdata in the name field.
  • Select Data/NDS from the Type menu.
  • Click the CREATE button. The Named Data Set Design page is shown.
  • Select HSQL from the Type menu.
  • Our data source will be limited to non-zero data. We sort it in descending order to make it easier to chart, but this isn’t necessary. Set the SQL to:select * from DataTable where ClickThru > 0 order by keyword desc
  • Cick TEST to verify the NDS. Close the test window and click DONE when finished.
     
    dev
     

Step 3 – Create the Charts

Create the Chart

  • From the Project Files list in our Cache Project, Click NEW.
  • Type cachechart.cdx into the name field. Select Chart from the Type menu. Click CREATE. This will display the Chart Design Wizard.
  • Set Title to Keyword Click Thru
  • Set Color Table to Earthy. (We will replace this in the CDL file in a moment in order to dynamically generate colored bars.)
  • Set Type to BarWidth to 600Height to 500.
  • At the bottom of the wizard, uncheck 3D Effect and set the Orientation to Horizontal.
  • Now we will begin to integrate the data. Select cachedata.ndx from the Data Source menu next to the Bars Data Parameter. This indicates that the bars are to use the cachedata.ndx file as the data source. In effect, however, this NDS file is really just a placeholder for the raw data we will store into the cache later.
  • The cachedata.ndx data source needs to be filtered properly to display bars in the chart. Click the DEFINE button. This will display the NDS Filter window. Note that our sample data is displayed. Uncheck Col 0Col 2, and Row 0. Click SAVE.
  • Select cachedata.ndx from the Data Source menu next to the AxisLabels Data Parameter. This indicates that the bar labels are to use the cachedata.ndx NDS file as the data source. Again, however, this NDS file is really just a placeholder.
  • The cachedata.ndx data source needs to be filtered properly to display bar labels in the chart. Click the DEFINE button. This will display the NDS Filter window. Note that our sample data is displayed. Click RESET to eliminate the previous settings. Then uncheck Col 1, Col 2, andRow 0. Click SAVE.
  • Click the TEST button, our chart should display our sample data as shown here:
     
    dev
     
  • Close the test window, and Click DONE in the chart wizard.
  • Click the Edit icon next to the cachechart.cdx item to bring up the page editor.
  • Find the lines that look like:
     

    ColorTable = x284b53,x005699,xb8bc9c,x271651,xaa0036,xecf0b9,x999966,x333366,xc3c3e6,x594330,
    xa0bdc4,x005699,x999966,x213321,xdace98;

    and replace with

    NdsParams = (“rowFilter”, “1*”), (“columnFilter”, “2”);
    ColorTable = NDS “cachedata.ndx”;

    Note that the NdsParams parameter is reading data from column 2, the color column.

  • Find the line that looks like:
     

    LeftTicLayout1 = (AUTO,0,2);

    This line tells the chart to autostagger labels if they get too close. Disable this feature by changing AUTO to NORMAL.

  • Click TEST. Now the chart will appear with the colors in the sample data. Close the test window and click DONE.

Step 4 – Writing to the Cache

In this step we will extract data from the data source, then modify and save the data for use by the chart.

NOTE: If writing to the cache to populate an element associated with an .ndx file on disk (e.g., a manual data set), make sure to first remove the cacheLifeSpan attribute from the .ndx or it will override the lifeSpan property passed in via the API.

Writing to the Cache in a JSP Environment

NOTE: Developers in remote JSP environments should first install the JSP toolkit as explained in the ‘Building A Simple Chart Application’ example. JSP deployment should be done in the appropriate manner for the remote environment. If developing in a remote JSP environment, create a blank file called “cachepage.jsp”.
NOTE: For example, Websphere, JBoss, Weblogic, JRun, etc. Developers creating JSP pages within the NetCharts Server do not need to install the toolkit.
    • From the Project Files list in our Cache Project, click the NEW button.
    • Type cachepage.jsp into the name field. Select Any from the Type menu to give us a blank page to start with. Click the CREATE button. This will produce a blank file.
    • Select the Edit icon next to the cachepage.jsp item.
    • Begin by adding the following import lines:
       

      <%@ page import=”java.util.*” %>
      <%@ page import=”netcharts.server.api.*” %>

    • Next, start the code block and create an instance of the NSWebToolKit that points to the project.
       

      <%
      NSWebToolKit toolKit = new NSWebToolKit(“Cache Project”);

NOTE: If on a remote JSP platform, make sure to use the NSWebToolKit API signature that allows passing of the servername, port number, and project.
  • Get the raw data. This will be returned in a two-dimensional object array.
     

    Object rawdata[][] = null;
    try {
    rawdata = toolKit.getNDSData(“rawdata.ndx”, null);
    } catch (Exception ex){
    String msg = “Can’t get raw data “+ex.getMessage();
    %>
    <%=msg%>
    <%
    return;
    }

  • Create a structure to hold the output data. Remember that our chart expects three columns,
    Keyword, Click Thru, and Color.
     

    Object cachedata[][] = new Object[rawdata.length][3];

  • Loop through the raw data copying the keyword and click thru rate, and then adding a color,
    based on the value of the click thru.
     

    for (int i=0; i < rawdata.length; i++){
    if (i==0){// column name row – put in our values and continue
    cachedata[i][0]=”Keyword”;
    cachedata[i][1]=”ClickThru”;
    cachedata[i][2]=”Color”;
    continue;
    }
     
    String keyword = (rawdata[i][0]).toString(); // keyword
    String ctstring = (rawdata[i][3]).toString(); // click thru
     
    Double clickthru = new Double(ctstring);
     
    cachedata[i][0] = keyword;
    cachedata[i][1] = clickthru;
    cachedata[i][2] = (clickthru.doubleValue() > 5.0)?”Navy”:”green”;
    }

  • Put our new data into the cache.
     

    try {
    toolKit.putDataCacheEntry(“AdCache”, cachedata, 10);
    } catch (Exception ex){
    String msg = “Can’t put data into cache:”+ex.getMessage();
    %>
    <%=msg%>
    <%
    return;
    }

  • Create the chart. Note that we set the special redraw flag to true. This forces a redraw since changes in the underlying NDS data will not cause the chart to refresh.
     

    String chart = null;
    try {
    Hashtable chartParams = new Hashtable();
    chartParams.put(“redraw”, “true”);
    chart = toolKit.getChartAsImageAndMap(“cachechart.cdx”, chartParams, request, true);
    } catch (Exception ex){
    chart = “Can’t create chart:”+netcharts.server.util.Util.formatForHtml(ex.getMessage());
    }

  • End the code block, and encapsulate the chart in the body tag.
     

    %>
    <html>
    <body>
    <%=chart%>
    </body>
    </html>

  • Click the TEST button or request the cachepage.jsp from the application server. The chart will appear as shown here. Note that all bars with a value over 5 are colored Navy blue and the others are green.
     
    dev
     

Writing to the Cache in a ASP Environment

  • Select the ASP link from the left panel, underneath API Toolkits. Download the ASP Toolkit off of the Developer page.
  • Unzip the toolkit and run the installation. This will install the files into the NetCharts Server install area in a new directory called “ToolKit”.
  • Set up a directory to hold the sample asp files. Note that the getncsimage.asp file, normally installed in C:\InetPub\wwwroot\, must be at the root level of the web directory or virtual directory that the ASP files utilizing the ASP Toolkit are stored in to function properly.
  • Run the NetCharts Server ASP ToolKit Configuration application from the Start menu and configure the default server and port.
  • Create an ASP in your remote environment called datapage.asp. Add the following lines to the ASP page:
     

    <html>
    <head>
    <title>Writing to the Cache using ASP</title>
    <%
    ‘ Create the toolkit.
    Dim toolKit
    Set toolKit = Server.CreateObject(“NetChartsServer.NSToolKit”)
    toolKit.setProject “Cache Project”
     
    ‘ Get the existing data.
    Dim objRec
    Set objRec = toolKit.getNDSData(“rawdata.ndx”)
     
    ‘ Create a new RecordSet to send the data back.
    Dim cloneRec
    Set cloneRec = Server.CreateObject(“ADODB.Recordset”)
    cloneRec.CursorLocation = 3 ‘ adUseClient
     
    ‘ Add just these three fields.
    cloneRec.Fields.Append “Keyword”, 200, 255 ‘ 200 = adVarChar
    cloneRec.Fields.Append “ClickThru”, 200, 255 ‘ 200 = adVarChar
    cloneRec.Fields.Append “Color”, 200, 255 ‘ 200 = adVarChar
     
    ‘ Open the RecordSet for use.
    cloneRec.Open
     
    ‘ Loop through the raw data copying the keyword and click
    ‘ thru rate, and then adding a color, based on the value
    ‘ of the click thru.
    While Not objRec.EOF
    Dim keyword, clickThruValue, color
    keyword = objRec(0)
    clickThruValue = objRec(3)
     
    If IsNumeric(clickThruValue) Then
    If (CDbl(clickThruValue) > 5.0) Then
    color = “Navy”
    Else
    color = “Green”
    End If
     
    ‘ Add a new record.
    cloneRec.AddNew
    cloneRec(0) = keyword
    cloneRec.Update
    cloneRec(1) = clickThruValue
    cloneRec.Update
    cloneRec(2) = color
    cloneRec.Update
    End If
     
    ‘ Move to the next existing record.
    objRec.MoveNext
    Wend
     
    ‘ Put our new data into the cache.
    Dim cacheKey
    cacheKey = “AdCache”
    toolKit.putDataCacheEntry cacheKey, cloneRec, 30
     
    ‘ Create the chart. Note that we set the special “redraw” flag to “true”.
    ‘ This forces a redraw since changes in the underlying NDS data will not
    ‘ cause the chart to refresh.
    Dim myHashTable
    Set myHashTable = Server.CreateObject(“NetChartsServer.Hashtable”)
    myHashTable.putValue “type”, “PNG”
    myHashTable.putValue “redraw”, “true”
     
    Dim chart
    chart = toolKit.getChartAsImageTag(“cachechart.cdx”, myHashTable)
    cloneRec.close
     
    objRec.close
    Set objRec = Nothing
    Set cloneRec = Nothing
    Set toolKit = Nothing
    %>
    </head>
    <body>
    <%=chart%>
    </body>
    </html>

  • Request datapage.asp. If configured properly, the chart is shown in the page.

Writing to the Cache in .NET

  • Select the .NET link from the Developer Console’s Control Panel, underneath API Toolkits.
  • Download the .NET Toolkit off of the Developer page.
  • Unzip the toolkit and run the installation.
  • Set up a directory to hold the sample ASP.NET files. Note that the getncsimage.aspx file (used for chart images) must be at the root level of the web directory or virtual directory that WebForms utilizing the .NET Toolkit are stored in to function properly.
  • Create a .NET WebForm in your remote environment called datapage.aspx. Add two Literal objects to the WebForm named çlientScript and theChart. Then add the following lines to the Page_Load method of the code behind page:
     

    Dim toolkit As NetChartsServer.NSWebToolKit = New NetChartsServer.NSWebToolKit(“Cache Project”, “localhost”, 8001)
     
    Try
    Dim dt As DataTable = toolkit.getNDSData(“rawdata.ndx”, Nothing, “MyTableName”)
    Dim cloneDT As DataTable = New DataTable(“MyTableName”)
     
    ‘ Add the columns
    Dim columns(3) As String
    columns(0) = “Keyword”
    columns(1) = “ClickThru”
    columns(2) = “Color”
     
    For i As Integer = 0 To columns.GetUpperBound(0)
    Dim dc As DataColumn = New DataColumn
    dc.DataType = System.Type.GetType(“System.String”)
    dc.ColumnName = columns(i)
    dc.AutoIncrement = False
    dc.Caption = columns(i)
    dc.ReadOnly = False
    dc.Unique = False
    cloneDT.Columns.Add(dc)
    Next
     
    ‘ Loop through the raw data copying the keyword and click
    ‘ thru rate, and then adding a color, based on the value
    ‘ of the click thru.
    For i As Integer = 1 To dt.Rows.Count – 1
    Dim oldRow As DataRow = dt.Rows(i)
    Dim dr As DataRow = cloneDT.NewRow()
    dr.Item(0) = oldRow.Item(0)
    dr.Item(1) = oldRow.Item(3)
     
    If (CDbl(oldRow.Item(3)) > 5.0) Then
    dr.Item(2) = “Navy”
    Else
    dr.Item(2) = “Green”
    End If
     
    cloneDT.Rows.Add(dr)
    Next
     
    ‘ Put our new data into the cache.
    toolkit.putDataCacheEntry(“AdCache”, cloneDT, 4)
     
    ‘ Create the chart. Note that we set the special “redraw” flag to “true”.
    Dim hashTable As Hashtable = New Hashtable
    hashTable.Add(“type”, “PNG”)
    hashTable.Add(“redraw”, “true”)
     
    theChart.Text = toolkit.getChartAsImageAndMap(“cachechart.cdx”, hashTable,””, False, Request.ApplicationPath & “/getncsimage.aspx”)
    Catch nce As NetChartsServer.NSToolKitException
    theChart.Text = nce.Message
    End Try
     
    clientScript.Text = toolkit.getRolloverJavaScript

  • Request datapage.aspx. If configured properly, the chart is shown in the page.