JSON-RPC
From NNTPGrab
As of NNTPGrab 0.6 it is possible to control NNTPGrab using JSON-RPC. This requires the JSON-RPC plugin to be loaded in the program.
[edit] Installation
JSON-RPC support is provided by the JSON-RPC plugin which is installed along with the program by default. To activate the JSON-RPC server, you need to go to the configuration tab in NNTPGrab, select Plugins and make sure that the option Enable embedded webserver is checked. It's also possible to indicate the port number of which the JSON-RPC server needs to listen.
After the JSON-RPC server has been activated you can send JSON-RPC commands to the URL https://localhost:5423/jsonrpc (where localhost is the hostname or IP address of the host where NNTPGrab is running and 5423 is the port number which has been set in the NNTPGrab configuration.
[edit] Handshake
[edit] JSON-RPC over HTTP
When using JSON-RPC no special handshake is needed. It's sufficient to send a regular HTTP POST request containing JSON-RPC data to the server. The connection will automatically be closed by the server after every request as Mongoose (the embedded webserver library used by NNTPGrab) doesn't support HTTP keep-alive.
[edit] JSON-RPC over TCP
If possible it's recommended to use JSON-RPC over TCP. This is because NNTPGrab can emit events (notifications) at any time. When using JSON-RPC over TCP the connection doesn't need to be re-established after every request so this saves time and resources.
For JSON-RPC over TCP the same port as JSON-RPC over HTTP can be used, but instead of a HTTP request, a custom request needs to be sent by the client to the server. This way the webserver knows what method needs to be used.
To activate JSON-RPC over TCP the client needs to send this message to the server:
NNTPGrab - API Version 20100109\r\n\r\n
The value 20100109 is the current API version as defined in the file nntpgrab_core/nntpgrab_plugins.h and the \r\n\r\n sequence are two newlines
In response, the server can send one of these replies:
OK\r\n API mismatch, expected 20091228\r\n
If the server sends the OK response then the handshake is completed and JSON-RPC request can be sent to the server. The server can also send any events (notifications) back to the client at any moment
If the server sends the API mismatch response then a different version of the API is expected. The value 20091228 mentioned here is just an example, it can be different depending on the version of NNTPGrab installed. The connection will be closed automatically by the server if this response is emit
[edit] Authentication
[edit] JSON-RPC over HTTP
Authentication is required to use JSON-RPC over HTTP. This is realized by the HTTP authentication method Digest
[edit] JSON-RPC over TCP
Authentication isn't implemented at the moment for JSON-RPC over TCP, but it will be in future versions. Check back later
[edit] Functions provided by the NNTPGrab JSON-RPC plugin
[edit] nntpgrab_get_version_info
nntpgrab_get_version_info - Returns several version numbers belonging to NNTPGrab
object nntpgrab_get_version_info
Returns an object with the following fields:
( [glue_version] [major_version] [minor_version] [micro_version] [version] )
Available since NNTPGrab 0.6
[edit] nntpgrab_config_get_avail_servers
nntpgrab_config_get_avail_servers - Retrieve a list of the available usenet servers
array nntpgrab_config_get_avail_servers()
Returns an array containing the servername's of all the available usenet servers
Available since NNTPGrab 0.6
[edit] nntpgrab_config_get_server_info
nntpgrab_config_get_server_info - Retrieve the configuration details about a specific usenet server
object nntpgrab_config_get_server_info(string servername)
When the given servername isn't known an error is returned, otherwise an object containing the following elements is returned
( [servername] [hostname] [port] [username] [password] [max_threads] [priority] [use_ssl] [enabled] )
The contents of the field priority has one of the following values:
0 - High priority 1 - Normal priority 2 - Low priority
Available since NNTPGrab 0.6
[edit] nntpgrab_config_add_server
nntpgrab_config_add_server - Add a new server to the NNTPGrab configuration
boolean nntpgrab_config_add_server(array serverconfig)
Returns TRUE when the server was successfully added to the NNTPGrab configuration or FALSE when the given servername is already known.
The serverconfig array needs to have the following fields:
Array ( [servername] [hostname] [port] [username] [password] [max_threads] [priority] [use_ssl] [enabled] )
The contents of the field priority needs to have one of the following values:
0 - High priority 1 - Normal priority 2 - Low priority
Available since NNTPGrab 0.6
[edit] nntpgrab_config_del_server
nntpgrab_config_del_server - Remove a server from the NNTPGrab configuration
boolean nntpgrab_config_del_server(string servername)
Returns TRUE when the server was successfully removed from the NNTPGrab configuration or FALSE when the given servername isn't known.
Available since NNTPGrab 0.6
[edit] nntpgrab_config_edit_server
nntpgrab_config_edit_server - Change the settings of a server
boolean nntpgrab_config_edit_server(string original_servername, object serverconfig)
Returns TRUE when the change was successfully performed or FALSE when the given original_servername isn't known.
The serverconfig object needs to have the following fields:
( [servername] [hostname] [port] [username] [password] [max_threads] [priority] [use_ssl] [enabled] )
The contents of the field priority needs to have one of the following values:
0 - High priority 1 - Normal priority 2 - Low priority
Available since NNTPGrab 0.6
[edit] nntpgrab_config_get_opts
nntpgrab_config_get_opts - Retrieve the basic configuration information from NNTPGrab
object nntpgrab_config_get_opts()
Returns an object containing the following fields:
( [download_directory] [temp_directory] [enable_intelligent_par2_downloading] [enable_par2_repair] [enable_auto_import] [auto_import_directory] [move_file_after_auto_import] [enable_auto_unpack] [enable_bandwidth_shaping] [max_bandwidth] [enable_webserver] [webserver_port] [enable_logger] )
Available since NNTPGrab 0.6
[edit] nntpgrab_config_set_opts
nntpgrab_config_set_opts - Change the basic configuration of NNTPGrab
boolean nntpgrab_config_set_opts(object options)
Returns TRUE when the changes have been performed or FALSE when there was an error
The options object needs to have the following fields:
( [download_directory] [temp_directory] [enable_intelligent_par2_downloading] [enable_par2_repair] [enable_auto_import] [auto_import_directory] [move_file_after_auto_import] [enable_auto_unpack] [enable_bandwidth_shaping] [max_bandwidth] [enable_webserver] [webserver_port] [enable_logger] )
Available since NNTPGrab 0.6
[edit] nntpgrab_schedular_start
nntpgrab_schedular_start - Start the NNTPGrab schedular
boolean nntpgrab_schedular_start()
Returns TRUE when the schedular has been successfully started or FALSE when the schedular was already running or in the stopping state
Available since NNTPGrab 0.6
[edit] nntpgrab_schedular_stop
nntpgrab_schedular_stop - Stop the NNTPGrab schedular
nntpgrab_schedular_stop(boolean wait)
If wait is FALSE, a request will be sent to the NNTPGrab Server to stop the schedular, but the function will not wait for the schedular to be really stopped. If wait is TRUE, this function will wait until the schedular is really stopped.
Available since NNTPGrab 0.6
[edit] nntpgrab_schedular_get_state
nntpgrab_schedular_get_state - Retrieve the current state of the NNTPGrab schedular
int nntpgrab_schedular_get_state()
This function can return one of the following values:
0 - Schedular is running 1 - Schedular is stopping 2 - Schedular is stopped
Available since NNTPGrab 0.6
[edit] nntpgrab_schedular_add_file_to_queue
nntpgrab_schedular_add_file_to_queue - Add a new file to the download queue
boolean nntpgrab_schedular_add_file_to_queue(string collection_name, string subject, string poster, int timestamp, int file_size, array groups, array parts)
Returns TRUE when the file has been successfully added or FALSE when there already is a file with the same subject in the same collection_name in the download queue
The file_size needs to be in bytes
The array groups contains a list of usenet groups in which this file is published
The array parts contains an array which has the following fields:
Array
(
[0] => Object -- This block is representing one part. This block can occur multiple times
(
[message_id]
[part_num]
[size]
)
)
Available since NNTPGrab 0.6
[edit] nntpgrab_schedular_del_file_from_queue
nntpgrab_schedular_del_file_from_queue - Remove a file from the download queue
boolean nntpgrab_schedular_del_file_from_queue(string collection_name, string subject)
Returns TRUE when the file has been successfully removed or FALSE when the given collection_name and subject aren't known
Available since NNTPGrab 0.6
[edit] nntpgrab_schedular_restart_file
nntpgrab_schedular_restart_file - Restart a file
boolean nntpgrab_schedular_restart_file(string collection_name, string subject)
Returns TRUE when the file has been successfully restarted or FALSE when the given collection_name and subject aren't known
Available since NNTPGrab 0.6
[edit] nntpgrab_schedular_save_queue
nntpgrab_schedular_save_queue - Save the download queue
boolean nntpgrab_schedular_save_queue()
This function should be called after several new tasks have been added to the download queue
Returns TRUE when the save has been successfull or FALSE when there was an error
Available since NNTPGrab 0.6
[edit] nntpgrab_schedular_get_all_tasks
nntpgrab_schedular_get_all_tasks - Retrieve all the tasks in the download queue
array nntpgrab_schedular_get_all_tasks()
The array returned from this function contains a lot of data:
Array
(
[0] => Object -- This block is representing one collection. This block can occur multiple times
(
[collection_name]
[total_size]
[total_size_remaining]
[files] => Array
(
[1] => Object -- This is representing one file. This block can occur multiple times
(
[subject]
[poster]
[stamp]
[file_size]
[file_size_remaining]
[num_parts_total]
[num_parts_downloaded]
[num_parts_failed]
[status]
[groups] => Array -- This is representing a list of usenet groups in which this file is published
(
[0]
)
)
)
)
)
The fields total_size and file_size mention the size in KB (kilobytes, 1024 bytes)
Possible values for the status are:
0 - Waiting for download 1 - Downloading 2 - Waiting for decoding 3 - Decoding 4 - Finished - complete 5 - Finished - incomplete 6 - Finished - no parts available 7 - Skipped
Available since NNTPGrab 0.6
[edit] nntpgrab_schedular_move_file
nntpgrab_schedular_move_file - Change the position and/or location of a file in the download queue
boolean nntpgrab_schedular_move_file(string collection_name_src, string subject_src, string collection_name_dest, int position_dest)
collection_name_src can be the same as collection_name_dest. If position_dest is -1, the file will be added to the end of the queue. If subject is empty, then position_dest will mean the position of all the collections in the download queue.
Available since NNTPGrab 0.6
[edit] nntpgrab_schedular_move_collection
nntpgrab_schedular_move_collection - Change the position of a collection in the download queue
boolean nntpgrab_schedular_move_collection(string collection_name, int new_position)
If new_position is -1, the file will be added to the end of the queue.
Available since NNTPGrab 0.6
[edit] nntpgrab_plugins_get_avail_plugins
nntpgrab_plugins_get_avail_plugins - Get a list of all the available plugins
array nntpgrab_plugins_get_avail_plugins()
Returns an array containing string's with the name of the plugin
Available since NNTPGrab 0.6
[edit] nntpgrab_plugins_get_plugin_info
nntpgrab_plugins_get_plugin_info - Retrieve information about a specific plugin
object nntpgrab_plugins_get_plugin_info(string plugin_name)
The returned object contains these fields:
( [name] [version] [author] [url] [description] [is_loaded] [is_persistent] )
Available since NNTPGrab 0.6
[edit] nntpgrab_plugins_load_plugin
nntpgrab_plugins_load_plugin - Load a plugin
boolean nntpgrab_plugins_load_plugin(string plugin_name)
Returns TRUE if the plugin was successfully loaded. In case of an error, an JSON error will be returned containing details
Available since NNTPGrab 0.6
[edit] nntpgrab_plugins_unload_plugin
nntpgrab_plugins_unload_plugin - Unload a plugin
boolean nntpgrab_plugins_unload_plugin(string plugin_name)
Returns TRUE if the plugin was successfully unloaded. In case of an error, an JSON error will be returned containing details
Available since NNTPGrab 0.6
[edit] nntpgrab_plugins_set_persistent
nntpgrab_plugin_set_persistent - Change the persistent flag of a plugin
void nntpgrab_plugins_set_persistent(string plugin_name, boolean is_persistent)
In case of an error, an JSON error will be returned containing details
Available since NNTPGrab 0.6
[edit] nntpgrab_utils_strip_subject
nntpgrab_utils_strip_subject - Split a subject in various useable fields
array nntpgrab_utils_strip_subject(string subject)
The array returned contains the following fields:
Array ( [subject] [subject_without_partnum] [file_num] [total_files] [filename] [extension] [file_type] [par2_startnum] [num_par2_blocks] [part_num] [total_parts] )
Note that some fields can be missing. If a field is missing, it can't be detected in the given subject
Available since NNTPGrab 0.6
[edit] nntpgrab_utils_calculate_file_size
nntpgrab_utils_calculate_file_size - Returns the given file size in a human readable notation
string nntpgrab_utils_calculate_file_size(long filesize)
The filesize should be in KB (kilobytes)
Available since NNTPGrab 0.6
[edit] nntpgrab_utils_calculate_estimated_time_remaining
nntpgrab_utils_calculate_estimated_time_remaining - Make an estimation when a file with the given size will be completed
int nntpgrab_utils_calculate_estimated_time_remaining (array speed_history, int filesize)
The array speed_history should be the history part of the array returned by the function nntpgrab_connections_get_speed_history The filesize is in KB (kilobytes) The value returned is the number of seconds necessary to complete the given file size
Available since NNTPGrab 0.6
[edit] nntpgrab_utils_get_readable_time_remaining
nntpgrab_utils_get_readable_time_remaining - Returns the human readable notation of a given number of seconds
string nntpgrab_utils_get_readable_time_remaining (int time_remaining)
The value time_remaining is returned by the function nntpgrab_utils_calculate_estimated_time_remaining
Available since NNTPGrab 0.6
[edit] nntpgrab_utils_get_readable_finished_time
nntpgrab_utils_get_readable_finished_time - Returns the human readable notation of a given number of seconds relative to the current time
string nntpgrab_utils_get_readable_finished_time (int time_remaining)
The value time_remaining is returned by the function nntpgrab_utils_calculate_estimated_time_remaining
Available since NNTPGrab 0.6
[edit] nntpgrab_utils_get_folder_listing
nntpgrab_utils_get_folder_listing - Retrieves a list containing all the sub-folders which are in the given folder
mixed nntpgrab_config_get_folder_listing(string parent)
Returns FALSE when the given parent isn't known by the NNTPGrab Server or an array containing object's with these values:
( [folder] [has_subfolders] )
Available since NNTPGrab 0.6
[edit] Events provided by the NNTPGrab JSON-RPC plugin
When connected to NNTPGrab using JSON-RPC over TCP events can be emit at any moment. When using JSON-RPC over HTTP, events (or in JSON-RPC terms: notifications) can be sent along with responses to requests. These are the possible events which the NNTPGrab Core can send:
[edit] config_changed
Something in the configuration has changed
config_changed
[edit] part_download_start
The download of a part has just started
part_download_start string servername, int conn_id, string collection_name, string subject, int part_num
[edit] part_done
The download of a part has completed successfully
part_done string servername, int conn_id, string collection_name, string subject, int part_num, int size
[edit] part_failed
The download of a part has failed (probably because the part isn't available on the server
part_failed string servername, int conn_id, string collection_name, string subject, int part_num, int size, boolean all_servers_tried
[edit] traffic_monitor_update
Every second, traffic statistics about the last 10 seconds are given
traffic_monitor_update int bytes_received1, int bytes_received2, int bytes_received3, int bytes_received4, int bytes_received5, int bytes_received6. int bytes_received7, int bytes_received8, int bytes_received9, int bytes_received10, int stamp, double average
[edit] part_progress_update
Every 1/10th second of a part download, this message will be sent
part_progress_update string servername, int conn_id, string subject, int part_num, int bytes_downloaded, int bytes_total
[edit] collection_added
A new collection was added
collection_added string collection_name, string poster
[edit] collection_removed
A collection was removed
collection_removed string collection_name
[edit] collection_modified
The poster of a collection was modified
collection_modified string collection_name, string poster
[edit] file_added
A file was added to a already existing collection
file_added string collection_name, string subject, string poster, int stamp, int file_size, int total_size, int total_size_remaining, int status, int num_parts, array groups
The groups argument is a list of newsgroups (string's)
NOTE: The fields file_size, total_size and total_size_remaining contain the number of KB's instead of bytes
[edit] file_removed
A file was removed from the download queue
file_removed string collection_name, string subject, int total_size, int total_size_remaining
NOTE: The fields total_size and total_size_remaining contain the number of KB's instead of bytes
[edit] file_download_state_update
The state of a file in the download queue has changed
file_download_state_update string collection_name, string subject, int num_parts_total, int num_parts_done, int num_parts_failed, int file_size, int file_size_remaining, int total_size, int total_size_remaining
NOTE: The fields file_size, file_size_remaining, total_size and total_size_remaining contain the number of KB's instead of bytes
[edit] file_state_changed
A file in the download queue was restarted
file_state_changed string collection_name, string subject, string real_filename, int old_state, int new_state
[edit] connection_connecting
A new connection is being made to a usenet server
connection_connecting string servername, int conn_id
[edit] connection_connected
A connection attempt has succeeded
connection_connected string servername, int conn_id, string welcome_msg
[edit] connection_disconnect
A connection to a usenet server was disconnected
connection_disconnect string servername, int conn_id, int disconnect_type, string reason
disconnect_type can be any of the following values:
0: normal 1: hostname could not be resolved 2: connection refused 3: too many connections 4: connection timeout 5: read error 6: read timeout 7: write error 8: idle timeout 9: invalid message received from server 10: login failure 11: unable to initialize SSL 12: unexpected error occured
[edit] schedular_state_changed
The state of the schedular has changed
schedular_state_changed int new_state, string reason
[edit] log_message
A log message has been emitted
log_message string component, int log_level, string msg
log_level can contain one of the following values:
1: info 2: warning 3: error 4: fatal 5: debug
[edit] task_moved
The position of a task in the download queue has changed
task_moved string orig_collection_name, string subject, string new_collection_name, int old_position, int new_position
[edit] collection_moved
The position of a collection in the download queue has changed
collection_moved string collection_name, int old_position, int new_position
[edit] all_downloads_completed
This message is emit when all the items in the download queue are completed
all_downloads_completed
[edit] plugin_loaded
A plugin has been loaded
plugin_loaded string plugin_name, boolean is_persistent
[edit] plugin_unloaded
A plugin has been unloaded
plugin_unloaded string plugin_name
[edit] plugin_event
A plugin has emit an event
plugin_event string plugin_name, string event_name, array values
The values parameter is an array containing string's
[edit] Plugin events
Certain plugin can emit events which are interesting to frontends. This is a list of all known events which can be emit by plugins (as part of the plugin_event event)
[edit] PAR2 plugin
The name of this plugin is PAR2
[edit] par2_begin_verify
A PAR2 verification (and if necessary repair) has just started
values[0]: collection_name values[1]: active_par2_filename
[edit] par2_load_progress_update
A file is now being verified
values[0]: collection_name values[1]: active_par2_filename values[2]: filename values[3]: progress
[edit] par2_recovery_file_loaded
A PAR2 recovery file has been loaded completely
values[0]: collection_name values[1]: active_par2_filename values[2]: filename values[3]: num_new_packets values[4]: num_blocks_found
[edit] par2_file_loaded
A regular file has been loaded completely
values[0]: collection_name values[1]: active_par2_filename values[2]: filename values[3]: state values[4]: num_blocks_found values[5]: num_blocks_expected
The field state can be one of the following values:
MISSING FOUND DAMAGED NO_NEW_BLOCKS_FOUND
[edit] par2_repair_progress_update
A PAR2 repair is now going on
values[0]: collection_name values[1]: active_par2_filename values[2]: progress
[edit] par2_repair_failure
The PAR2 repair has failed because there aren't enough recovery blocks available or some other error
values[0]: collection_name values[1]: active_par2_filename values[2]: error message values[3]: num_blocks_more_required
[edit] par2_repair_success
The PAR2 repair has completed successfully
values[0]: collection_name values[1]: active_par2_filename
[edit] par2_no_repair_required
All the files in the PAR2 set have been verified and no repair is necessary
values[0]: collection_name values[1]: active_par2_filename
[edit] Unpack plugin
The name of this plugin is Unpack
[edit] unpack_progress_update
The unpack component has made some progress
values[0]: collection_name values[1]: filename values[2]: progress
[edit] unpack_message_received
The unpack component has emit a message. This message mostly is the file which is currently extracted
values[0]: collection_name values[1]: filename values[2]: message
[edit] unpack_working_archive_changed
The unpack component is now unpacking from a new archive file
values[0]: collection_name values[1]: filename values[2]: new archive filename
