This object allows one to display the results for training and testing at different time points in the experiment, i.e., temporal cross-training (TCT) results, for results that were created using the standard_resample_CV.run_cv_decoding method. The results are plotted in an imagesc matrix where the y-axis indicates the time the classifier was trained and the x-axis indicates the time when the classifier was tested. Additionally, the results can be displayed as a movie, with each frame showing the results for training the classifier at one particular time (indicated by a line segment at the bottom of the screen), and testing the classifier at all other times (and the results for training and testing at the same time as displayed as a black curve on the figure for reference).
Methods
plot_obj = plot_standard_results_TCT_object(‘result_file_name’)
The constructor. The input parameter for this constructor, result_file_name, is a string that has the name of a file that has the saved results in ‘standard format’ (i.e., results that were generated by running the standard_resample_CV.run_cv_decoding method). Alternatively, the constructor can take a [num_training_times x num_test_times] matrix with precompiled decoding results and will display a TCT plot with these precompiled results. This can be useful, for example, if one wants to average several results together from different decoding experiments and then display the average results as a TCT plot, etc.
min_and_max_data_range = plot_obj.plot_results
The main method which creates a figure that displays the temporal cross-training result matrix, and also optionally displays a movie showing the decoding results for when the classifier was trained with data at different points in time. The method also returns the minimum and maximum value of that data in the variable min_and_max_data_range which is useful if one wants to plot multiple results with the same color range (see the color_result_range property below). As described below, there are several optional properties that can be set prior to calling the plot_results method that will change how the results are displayed.
Properties
result_type_to_plot (default = 1). Specifies which type of decoding result should be plotted.
If this is set to 1, the zero-one loss results are plotted.
If this is set to 2, the normalized rank results are plotted.
If this is set to 3, the mean decision values are plotted.
If this is set to 4, the ROC_AUC results run separately on each cross-validation split are plotted.
If this is set to 5, the ROC_AUC results combined over cross-validation splits are plotted.
If this is set to 6, the mutual information created from a confusion matrix that combining data from all resamples runs is plotted.
If this is set to 7, mutual information created from a confusion matrix that is calculated separately and then averaged over resample runs is plotted.
plot_time_intervals (default [])
This property specifies which time points in the experiment the results correspond to (i.e,. this specifies the x-axis values that the results are plotting against). If this value is set to a vector, then all results specified in result_file_names will use the same time range. Alternatively this property can be set with the following fields plot_time_intervals.bin_width, plot_time_intervals.sampling_interval, and optionally plot_time_intervals.alignment_event_time and plot_time_intervals.alignment_type, which will create a time interval that has the corresponding bin width, sampling interval and and alignment time which specified where 0 on the x-axis will be. The field alignment_type specifies whether the bins should be centered (default value of 2), whether they should be aligned to the end of the bins (value of 1) or whether they should be aligned to the start of the bin (value of 3). If this property is left empty (default value), then the plot_object attempts to use the binning parameters for each file using the saved in the DECODING_RESULTS.DS_PARAMETERS.binned_site_info.binning_parameters field which was originally created by the create_binned_data_from_raster_data function and pass through the DS and CV objects.
plot_inds
Plots a smaller range of the decoding results that are given by the indices listed in this vector.
saved_results_structure_name
By default this object assumes that all results were saved in a structure called DECODING_RESULTS. If the structure of the saved results has a different name, the name can be specified here and the results will be plotted using this name.
color_result_range
Specifies the range of the colormap for showing worst to best results.
figure_position (default = [])
Specifies the position of where the figure should be plotted. This can be useful because resizing the figure after it is plotted can potentially cause errors in the tick labels. If a figure is open to a particular size prior to running this function and this property is left empty, the function will use the size of the current figure.
TCT_figure_number (default = 1)
The figure number to display the TCT matrix.
plot_colorbar (default = 1)
If this is set to zero, the colorbar will not be plotted.
significant_event_times
This will cause vertical lines to be drawn at the times specified in this vector which can be used to indicate significant events that occurred during a trial.
significant_training_event_times
This will cause horizontal lines to be drawn at the times specified in this vector which can be used to indicate significant events that occurred during particular training times in a trial. [added in NDT version 1.4]
decoding_result_type_name (default = [])
Sets the name of result type chosen. If this is left empty, the name will be chosen based on the type of result_type used. If displaying a movie of the results, this property will also be used for the label y-axis of the movie.
font_size (default = 16)
The font size for the axis labels.
ylabel_name (default = ‘Train time (ms)’
The label of the y-axis.
xlabel_name (default = ‘Test Time (ms)’ )
The label of the x-axis.
plot_training_latencies_increasing_up_the_y_axis (default = 1).
If this is set to 1, then the training latencies that are earlier in the trial will be plotted closer to the x-axis (i.e., the results will be equivalent to using ‘axis xy’ rather than ‘axis ij’)
display_TCT_movie (default = 1)
If this is set to 1, then a click through of the TCT results will be shown with a blue line indicating the classification accuracy for training the classifier at one point in time (indicated by a blue circle below the plot) and testing at all other points in time. Also plotted are the usual decoding results for training and testing at the same time (black line) as a reference. Every time a button is pressed, the blue line updates showing the results for training at the next point in time. When this property is set to 0, these movies are not shown, which is useful when you want to just created figures showing just the TCT matrix.
movie_figure_number (default = 2)
The figure number that the TCT movie will be shown in.
Properties for TCT movie
To display movies of the TCT results, where each frame shows the results for training at t1 and testing at t2, the display_TCT_movie must be set to one (which is the default behavior of this property). If plot_obj.display_TCT_movie = 1 then the following additional properties can be set:
display_movie_pause_time (default = -1)
How long each frame of the movie should be displayed for. If this is set to a value less than one, then the frame will be shown until a key is pressed.
movie_figure_number (default = 2)
The figure number that the TCT movie will be shown in.
movie_save_name (default = [])
If this property is set to a string, the movie will be saved as an avi file using this name.
movie_time_period_titles (default = [])
This field allows the movie to display different titles at different time points (which is useful if one wants to give information about particular events such as ‘stimulus on’, ‘stimulus off’, etc.). If this field exists and if there is a subfields movie_time_period_titles.title_start_times that lists an array of times specifying when particular figure titles should be displayed, and if movie_time_period_titles.title_names is a cell array that lists the names of the titles at particular times, then the title of the movie figure will change to display these titles at the given times. Note that movie_time_period_titles.title_start_times and movie_time_period_titles.title_names must be the same length.
chance_level
Draws a horizontal line at the specified chance decoding level. If this is unspecified, then the change level will be 1/num_classes for 0-1 loss results, no line will be drawn for decision values, and .5 line will be drawn for normalized rank and ROC measures, and 0 will be drawn for mutual information results.
line_width (default = 2)
The line width of the plotted results.
sliding_result_color (default = ‘b’)
The color of the results for the TCT movie plot. This color is also used for indicating which interval was the training time that is shown at the bottom of the plot.
errorbar_file_name
If this is set to a string that contains the name of a file that has results in ‘standard format’, it will plot errorbars using the data from the file name listed.
errorbar_type_to_plot (default = 1)
If errorbar_file_name is specified, this field specified which type of decoding variance measure should be plotted. The values for this property are:
- If this is set to 1, then the standard deviations over resample runs (i.e., stdev.over_resamples) is used.
- If this is set to 2, then the standard deviations over the mean decoding results from each cross-validation split (i.e., stdev.over_CVs) is used. The mean value over all resample trials is then plotted.
- If this is set to 3, then the standard deviations over the mean decoding results from each cross-validation split combined together from all resample runs (i.e., over_CVs_combined_over_resamples) is used.
- If this is set to 4, then the standard deviation of individual results (e.g., 0 1 values if the 0-1 loss is used) from each cross-validation split (all_single_CV_vals) are used. The mean value over all resample runs and CV splits is plotted.
- If this is set to 5, then the standard deviation of individual results (e.g., 0 1 values if the 0-1 loss is used) from each CV split combined from all cross-validation splits in a single resample run (all_single_CV_vals_combined) are used. The mean value over all resample runs is plotted.
- If this is set to 6, and ROC_AUC results are plotted, then the standard deviation of the ROC results from over the different classes (over_classes) are used. The mean results over resample runs (and CV splits for the separate_CV_ROC_results) are plotted. This value can only be set when ROC_AUC values are plotted (i.e., result_type_to_plot = 4 or 5).
It should also be noted that if separate_CV_ROC_results plotted (i.e., result_type_to_plot = 4), then errorbar_type_to_plot can only be set to values of 1, 2, or 6 and if combined_CV_ROC_results plotted (i.e., result_type_to_plot = 5), then errorbar_type_to_plot can only be set to values of 1 or 6. Also if mutual information created from a confusion matrix that combining data is plotted, then one can not plot errorbars, and if mutual information created from a confusion matrix that is calculated separately and then averaged over resamples is plotted, then errorbar_type_to_plot can only be set to values of 1.
errorbar_stdev_multiplication_factor (default = 1)
When plotting the errorbars, the default behavior is to plot plus and minus 1 standard deviation for the standard deviation type that is specified. If errorbar_stdev_multiplication_factor is set to a value of k, then the errorbars will be plotted as mean_results plus/minus k times the standard deviation type is plotted.
errorbar_transparency_level (default = .2)
Sets the transparency level of the errorbars.
errorbar_edge_transparency_level (default = .2)
Sets the transparency level of the edges of the errorbars.