Managing Data

If you are working with large datasets, you probably want to keep them on the server. Typically, the server has the memory and storage resources that large datasets require. On the other hand, client machines typically have limited memory and storage capacity. Furthermore, the transfer of large datasets between the client and server (for example, across the Internet) can consume network resources.

JWAVE addresses this problem with data proxies. A proxy is an object-oriented design term that refers to a surrogate or placeholder that controls access to another object. A data proxy, then, is a surrogate object that refers to data.

If the data is, for instance, a large multidimensional array stored in a PV-WAVE session, the client can use a proxy object to refer to that data. Using a proxy object, the client can request operations to be performed on the data, such as:

• running an analysis or filtering program on the data

• copying the data

• destroying the data

• renaming the data

• retrieving the data (returning it to the client)

All of these operations are controlled from the client while the data physically resides on the server, which greatly reduces the burden on client and network resources. Unless data is explicitly retrieved, it never has to be downloaded to the client.

What is a JWAVE Data Proxy

A Proxy class is an interface that describes a data proxy. The subclass of the Proxy class used by JWAVE clients to refer to data that is stored on the server is JWaveDataProxy.

Usually, a JWAVE wrapper function processes data in some way and stores the results. The JWAVE client may wish to use another JWAVE wrapper to perform further analysis on the stored results. To do this, the client needs to be able to reference the data stored on the server. The most efficient way to do this is to ask the server to return a data proxy, which the client can then use in the setParam method of subsequent function calls from the first wrapper.

While the end result of this processing might be some kind of plot or image that is sent back to the client, the important point to remember is that the actual data never has to leave the server.

How the data arrives on the server in the first place is up to the JWAVE application developer. The data could have originated on the client and been uploaded to the server. Or, the data could have been loaded on the server from a file or database, or calculated by a JWAVE wrapper function.

The data referred to by a JWaveDataProxy object persists on the server until it is explicitly deleted or the PV-WAVE session is shut down.

A JWaveDataProxy object is instantiated with:

• a valid connection to the server (a JWaveConnection object)

• a name by which to identify the data on the server, and

• a domain name (used to group datasets)

For instance, one way to construct a JWaveDataProxy object is:

new JWaveDataProxy(myconnection, 'MYDATA', 'MYDOMAIN')

where myconnection is the previously instantiated JWaveConnection object (tells the proxy how to find the server), MYDATA is the name of the data stored on the server, and MYDOMAIN is the domain.

When you construct a JWaveDataProxy object in this way, you are only providing the name of the data. This JWaveDataProxy object refers to a place to store data, but there is no guarantee that there is actually data there. If you do not know that there is already data on the server, you can use the Proxy.store method to send data to the PV-WAVE session.

There are two other ways to create a JWaveDataProxy object, and they both are used in conjunction with a JWaveExecute object.

The first way is to supply a ServerDataID object (which encapsulates the name and domain of a JWaveDataProxy object) when you use the setParam method.

setParam(String name, Object value, ServerDataID dataName) 

This associates values with the parameter name, as usual, but additionally the server stores the data as named by the ServerDataID object when it is sent with the next execute call.

This technique is useful when you have some data on the client, and you want to use that same dataset with several JWAVE wrapper functions. So you use the above setParam method to save the data on the first call, and then you use the following setParam on subsequent calls to associate a named parameter with that stored data:

setParam(String name, ServerDataID dataName) 

This technique stores data on the server, even though you are not manipulating JWaveDataProxy objects directly. If you need direct access to the data (like to delete it when you are done) you can use the same ServerDataID object that identifies the data name in a constructor to a new JWaveDataProxy object.

Another way to create a data proxy is when data is returned by a JWAVE wrapper function. Normally, the JWAVE wrapper returns values back to the client, but you can change this. Before you call the execute method, you can use the setReturnParamMode method to change how returned data will be handled.

There are a couple of overloaded methods for setReturnParamMode, but the primary one is:

 setReturnParamMode(String name, boolean returnVals, ServerDataID 
dataName)

If you use this method, and specify a ServerDataID object for the data name, then that parameter will be stored on the server when the JWAVE wrapper function returns. You can also use the boolean flag with one of the values JWaveExecute.RETURN_NO_VALUES or JWaveExecute.RETURN_VALUES. If you set this flag to RETURN_NO_VALUES, then no values are returned, but a JWaveDataProxy object to the stored data is returned.

This technique is useful for "chaining" the output of one JWAVE wrapper to the input of another, as you will see in a later example.

There are other ways to use setReturnParamMode object. If you set the flag to RETURN_VALUES and you set a ServerDataID object, then the data is stored and also returned to the client. You can even discard data that you do not want by setting RETURN_NO_VALUES without a ServerDataID object.

---

TIP: For detailed information on all of the constructors, variables, and methods of the JWaveDataProxy class, refer to the Javadoc reference. For information on Javadoc, see Using the JWAVE Javadoc Reference.

---

The Efficiency of Using Data Proxies

This section illustrates the efficiency of using data proxies in JWAVE applications.

In one typical JWAVE scenario, the client asks the server (a JWAVE wrapper function) to load or generate some data and then asks another JWAVE wrapper to process the data and return a Viewable object back to the client.

Figure 6-2 illustrates an inefficient version of this scenario, where data makes two complete round trips between the client and the server. This scenario can be summarized as follows:

• A JWaveExecute object executes a JWAVE wrapper function called DATAGEN on the PV-WAVE server.

• The generated data is then returned by the JWAVE wrapper to the client.

• The client passes the data to another JWaveExecute object, which sends it back to the server as input to another JWAVE wrapper called FILTER. (The data has made one round trip.)

• The filtered data is returned by the JWAVE wrapper to the client.

• The client passes the data to a JWaveView object, which sends it back to the server as input to another PV-WAVE routine called GRAPHICS. (The data has now made two round trips.)

• Finally, a plot is passed back to the client and displayed.

Figure 6-3 depicts the same type of processing as the previous figure; however, this time the client uses data proxies to refer to the data on the server. The actual dataset is never physically returned to the client. Only after processing is completed is an image object returned to the client and displayed.

In this case, note that the object returned to the client each time is a data proxy, rather than actual data. The client uses the proxy object in subsequent JWaveExecute methods (where, in the previous (inefficient) model, the actual data was passed).

By default, a JWAVE wrapper function returns physical data to the client. The client has to specify that it wants the JWAVE wrapper to return a proxy object. To do this, the client uses the JWaveExecute.setReturnParamMode method.

In the next section, we present an example illustrating the use of data proxies.

Example: Using Data Proxies

As previously noted, sending large datasets back and forth between client and server can be expensive in terms of network and client resources. In many cases, it is often more efficient to leave data on the server for at least part of the processing cycle.

This example discusses a JWAVE application that:

• creates an array of data on the client

• sends the data to the server where it is "processed" by a JWAVE wrapper function

• tells the server to return a proxy object rather than the actual data

• requests the server to execute a different JWAVE wrapper function to process the data further

• returns the actual processed data to the client and prints it

Figure 6-4 illustrates the flow of this JWAVE application.

The Java Client Program

import com.visualnumerics.jwave.*;

public class ProxyArray {
   public static void main(String[] args) {

      try {

  // Create a simple array as data
  float[] arr = new float[10];
  for (int i=0; iarr.length; ++i)
     arr[i] = i;

  // Connect to JWave server
  JWaveExecute command = new JWaveExecute("PROX1");

  // Set the data as ARRAY1 parameter
  command.setParam("ARRAY1", arr);

  // Ask that the result data (named DATA) be stored on the server,
  // and only return a Proxy
  ServerDataID saveData =  new ServerDataID("SAVE_NAME");

  command.setReturnParamMode("DATA", JWaveExecute.RETURN_NO_VALUES,
        saveData);

  // Execute PROX1
  command.execute();

  // Switch to new wrapper function
  command.setFunction("PROX2");

  // Clear previous parameters and return modes
  command.clearParams();
  command.clearReturnParamModes();

  // Set input to PROX2 to use data stored by PROX1
  // Could also use getReturnProxy("DATA") here rather than saveData
  command.setParam("ARRAY2", saveData);

  // Execute PROX2 and get result
  command.execute();
  float[] answer = (float[]) command.getReturnData("DATA");

  // Print result
  for (int i=0; ianswer.length; ++i)
     System.out.println(answer[i]);

      } catch (Exception e) {
  // Report problems
  System.out.println(e.toString());
      }
   }
}

Example 6-2 prox1.pro: Receives an array from the client and multiplies the elements by 1.5. The client asks the server to return a data proxy rather than the actual data.

FUNCTION PROX1, client_data

   ; Unpack data and parameters sent from the client. 
   arr =    GETPARAM(client_data, 'ARRAY1', /Value, Default=11)

   ; Change the array.
   mydata = arr * 1.5

   ; Return the changed array.
   RETURN, mydata
END

Example 6-3 prox2.pro: The client asks this server program to process the array it stored previously. The client refers to the data on the server with a proxy. Finally, the client asks the server to send back the actual data so that it can be printed. (This wrapper multiplies the previously stored array by 100.5.)

FUNCTION PROX2, client_data

   ; Unpack data and parameters sent from the client. 
   arr =    GETPARAM(client_data, 'ARRAY2', /Value, Default=11)

   ; Change the array.
   mydata = arr * 100.5

   ; Return the changed array.
   RETURN, mydata
END

Java Client Output

When the Java program in Example 6-1 is executed, the following output is printed on the client. (The numbers result from multiplying the first array by 1.5 and multiplying the resulting array by 100.5.)

% java proxyarr 
0.0
150.75
301.5
452.25
603.0
753.75
904.5
1055.25
1206.0
1356.75

Data Proxies Are Controlled by the Client

Note that in this example there is no special code in the JWAVE wrapper functions to enable the storing of data as proxies. The use of proxy data is totally under client control, and you do not need to do anything special in the wrapper functions to control this. This client-side control of proxies allows you to build JWAVE wrappers as modules, without planning all of the possible uses for them. As you build the client Java application, these JWAVE wrapper modules can then be "hooked together" in the best possible ways.

How Long is Proxy Data Stored on the Server

Data that is referred to by a proxy object persists for the duration of the PV-WAVE session in which it is stored. When the session ends, the data is lost.

---

TIP: You can use the Data Manager routines DMSave and DMRestore to store data between sessions. These JWAVE wrapper procedures are described in Appendix A, JWAVE Wrapper API.

---

Summary

A proxy is an object that represents data stored on the server (in PV-WAVE). Use data proxies to make JWAVE applications more efficient. By using proxies to refer to server data, you can reduce the burden on client and network resources.