History

Protocol¶

NOTE: This documentation is obsolete and doesn't apply anymore to NNTPGrab 0.6.0 and higher. For documentation about remote controlling NNTPGrab 0.6.0 and higher, please see the JSON-RPC documentation

This document describes how any frontend can communicate to the backend over raw sockets. This makes it possible to control the NNTPGrab backend from another host.

The NNTPGrab Core listens on port 5423/tcp.

After making a connection to this port, the NNTPGrab Core replies with the following welcome message:

NNTPGrab Server - API version 20090505

(where 20090505 is the version number of the protocol. This should match the clients expectations)

The documentation on this wiki is all based on API version 20090505

The NNTPGrab protocol is a line based protocol. Every command should end with the newline character (\n).

When connecting, the client should send one of the following commands before further communication is possible:

MODE LISTENER

MODE COMMAND

The server should return the following response:

OK

After this command, the socket is either broadcasting all the signals coming from the NNTPGrab Core or awaiting commands.

When an invalid command is entered, the server will send the response:

INVALID COMMAND

MODE COMMAND¶

CONFIG_GET_AVAIL_SERVERS¶

Returns a list of all the available usenet servers.

Notation:

CONFIG_GET_AVAIL_SERVERS

Response:

<servername1>
<servername2>
NULL

Example:

> CONFIG_GET_AVAIL_SERVERS
< NewsZilla
< NULL

CONFIG_GET_SERVER_INFO¶

Retrieve information about a specific usenet server. Returns NULL when the given servername isn't found

Notation:

CONFIG_GET_SERVER_INFO <servername>

Response:

NULL

<servername>
<hostname>
<port>
<username>
<password>
<max_threads>
<priority>
<use_ssl>
<enabled>
NULL

Example:

> CONFIG_GET_SERVER_INFO NewsZilla
< NewsZilla
< newszilla.xs4all.nl
< 119
< 
< 
< 3
< 1
< FALSE
< TRUE
< NULL

CONFIG_ADD_SERVER¶

Add a new usenet server
Returns FALSE when the servername already exists

Notation:

CONFIG_ADD_SERVER <servername>\t<hostname>\t<port>\t<username>\t<password>\t<max_threads>\t<priority>\t<use_ssl>\t<enabled>

Response:

TRUE

FALSE

Example:

> CONFIG_ADD_SERVER NewsZilla    newszilla.xs4all.nl    119            3    1    FALSE    TRUE
< TRUE

CONFIG_DEL_SERVER¶

Remove a usenet server
Returns FALSE when the servername isn't known

Notation:

CONFIG_DEL_SERVER <servername>

Response:

TRUE

FALSE

Example:

> CONFIG_DEL_SERVER NewsZilla
< TRUE

CONFIG_EDIT_SERVER¶

Changes the settings of a usenet server
Returns FALSE when the servername isn't known

Notation:

CONFIG_EDIT_SERVER <old_servername>\t<new_servername>\t<hostname>\t<port>\t<username>\t<password>\t<max_threads>\t<priority>\t<use_ssl>\t<enabled>

Response:

TRUE

FALSE

Example:

> CONFIG_EDIT_SERVER NewsZilla    NewsZilla_IPV6    newszilla6.xs4all.nl    119    my_username    my_pass    3    1    FALSE    TRUE
< TRUE

CONFIG_GET_OPTS¶

Retrieves the global settings of the NNTPGrab Core

Notation:

CONFIG_GET_OPTS

Response:

<download_directory>
<temp_directory>
<enable_intelligent_par2_downloading>
<enable_par2_repair>
<enable_auto_import>
<auto_import_directory>
<move_file_after_auto_import>
<enable_auto_unpack>
NULL

Example:

> CONFIG_GET_OPTS
< /home/test/NNTPGrab/Downloads
< /home/test/NNTPGrab/Temp
< FALSE
< TRUE
< TRUE
< /home/test/NNTPGrab/NZB
< TRUE
< TRUE
< NULL

CONFIG_SET_OPTS¶

Changes the global settings of the NNTPGrab Core
Returns FALSE when one of the new settings are invalid

Notation:

CONFIG_SET_OPTS <download_directory>\t<temp_directory>\t<enable_intelligent_par2_downloading>\t<enable_par2_repair>\t<enable_auto_import>\t<auto_import_directory>\t<move_file_after_auto_import>\t<enable_auto_unpack>

Response:

TRUE

FALSE

Example:

> CONFIG_SET_OPTS /home/test/NNTPGrab/Downloads    /home/test/NNTPGrab/Temp    FALSE    TRUE    TRUE    /home/test/NNTPGrab/NZB    FALSE    TRUE
< TRUE

CONFIG_GET_FOLDER_LISTING¶

Retrieves a list containing all the sub-folders which are in the given folder.
Each result also contains a TRUE or FALSE mentioning if the sub-folders itself contains sub-folders

Notation:

CONFIG_GET_FOLDER_LISTING <parent>

Response:

FALSE

NULL

folder1\tTRUE
folderX\tFALSE
NULL

Example:

> CONFIG_GET_FOLDER_LISTING /
< bin    FALSE
< etc    TRUE
< proc    TRUE
< sbin    FALSE
< usr    TRUE
< var    TRUE
< NULL

SCHEDULAR_START¶

Start the NNTPGrab schedular
Returns FALSE when the schedular is already running

Notation:

SCHEDULAR_START

Response:

TRUE

FALSE

Example:

> SCHEDULAR_START
< TRUE

SCHEDULAR_STOP¶

Stop the NNTPGrab schedular
Returns FALSE when the schedular is already stopped

Notation:

SCHEDULAR_STOP <wait>

Response:

TRUE

FALSE

Example:

> SCHEDULAR_STOP TRUE
< TRUE

SCHEDULAR_GET_STATE¶

Retrieves the current state of the schedular

Notation:

SCHEDULAR_GET_STATE

Response:

RUNNING

STOPPING

STOPPED

Example:

> SCHEDULAR_GET_STATE
< STOPPED

SCHEDULAR_ADD_TASK_TO_QUEUE¶

Add a new task (file) to the download queue
When a file contains multiple groups or multiple parts, these need to be split using the | character. A part needs to be in the notation <message_id>,<partnum>,<size>. The file size is mentioned in bytes and the stamp is an unix timestamp

Notation:

SCHEDULAR_ADD_TASK_TO_QUEUE <collection_name>\t<subject>\t<poster>\t<stamp>\t<file_size>\t<groups>\t<parts>

Response:

TRUE

FALSE <reason>

Example:

> SCHEDULAR_ADD_TASK_TO_QUEUE my collection    my subject    some poster    1199389869    123456789    alt.binaries.boneless|alt.binaries.nl    <some_msg@id>,1,500|<some_other@msg_id>,2,250
< FALSE The file already exists in the download queue

SCHEDULAR_DEL_TASK_FROM_QUEUE¶

Removes a task from the download queue

Notation:

SCHEDULAR_DEL_TASK_FROM_QUEUE <collection_name>\t<subject>

Response:

TRUE

FALSE <reason>

Example:

> SCHEDULAR_DEL_TASK_FROM_QUEUE my collection    my subject
< TRUE

SCHEDULAR_RESTART_TASK¶

Restarts a task already present in the download queue

Notation:

SCHEDULAR_RESTART_TASK <collection_name>\t<subject>

Response:

TRUE

FALSE <reason>

Example:

> SCHEDULAR_RESTART my collection    some other subject
< FALSE File is not present in the given collection

SCHEDULAR_SAVE_QUEUE¶

Saves the download queue. This needs to be done after added new files to the download queue

Notation:

SCHEDULAR_SAVE_QUEUE

Response:

TRUE

FALSE <reason>

Example:

> SCHEDULAR_SAVE_QUEUE
< TRUE

SCHEDULAR_FOREACH_TASK¶

Returns all the collections and tasks present in the download queue

Notation:

SCHEDULAR_FOREACH_TASK

Response:

COLLECTION <collection_name>\t<total_size>\t<total_size_remaining>\t<position>
\tFILE <subject>\t<poster>\t<stamp>\t<file_size>\t<file_size_remaining>\t<position>\t<num_parts_total>\t<num_parts_downloaded>\t<num_parts_failed>\t<status>\t<groups>
(this list can be returned multiple times)
NULL

Possible status'es:

WAITING_FOR_DOWNLOAD
DOWNLOADING
WAITING_FOR_DECODE
DECODING
FINISHED_COMPLETE
FINISHED_INCOMPLETE
FINISHED_NO_PARTS_AVAIL
SKIPPED

Example:

> SCHEDULAR_FOREACH_TASK
< COLLECTION my collection    12345678    100    3
<     FILE my_subject    some_poster    1199389869    0    1234567    1    100    10    0    WAITING_FOR_DOWNLOAD    alt.binaries.boneless|alt.binaries.nl
<     FILE some_other_subject    some poster    1199389900    100    565432    2    50    0    0    alt.binaries.x
< COLLECTION some other collection    626832    200
<     FILE another subject    another poster    1199331624    1583632    200    1    10    0    0    alt.binaries.x
< NULL

SCHEDULAR_MOVE_TASK¶

Change the position and/or location of a task in the download queue.
The collection_name_src can be the same as collection_name_dest. If the position is -1, the file will be added to the end of the queue. If subject is empty, then the position will mean the position of all the collections in the download queue.

Notation:

SCHEDULAR_MOVE_TASK <collection_name_src>\t<subject>\t<collection_name_dest>\t<position>

Response:

TRUE

FALSE

Example, change the position of a task within the same collection:

> SCHEDULAR_MOVE_TASK my collection    my subject    my collection    5
< TRUE

Example, move the task to another collection

> SCHEDULAR_MOVE_TASK my collection    my subject    another collection    -1
< FALSE

SCHEDULAR_MOVE_COLLECTION¶

Change the position of a collection in the download queue.
If the new_position is -1, the collection will be moved to the end of the queue.

Notation:

SCHEDULAR_MOVE_COLLECTION <collection_name>\t<new_position>

Response:

TRUE

FALSE

Example

> SCHEDULAR_MOVE_COLLECTION my collection    -1
< TRUE

MODE LISTENER¶

While in the listener mode, all signals coming from the NNTPGrab Core are automatically broadcasted on the socket. These are the possible messages which the NNTPGrab Core can send.

CONFIG_CHANGED¶

Something in the configuration has changed

CONFIG_CHANGED

PART_DOWNLOAD_START¶

The download of a part has just started

PART_DOWNLOAD_START <servername>\t<conn_id>\t<collection_name>\t<subject>\t<part_num>

PART_DONE¶

The download of a part has completed successfully

PART_DONE <servername>\t<conn_id>\t<collection_name>\t<subject>\t<part_num>\t<size>

PART_FAILED¶

The download of a part has failed (probably because the part isn't available on the server

PART_FAILED <servername>\t<conn_id>\t<collection_name>\t<subject>\t<part_num>\t<size>\t<all_servers_tried>

TRAFFIC_MONITOR_UPDATE¶

Every second, traffic statistics about the last 10 seconds are given

TRAFFIC_MONITOR_UPDATE <bytes_received1>\t<bytes_received2>\t<bytes_received3>\t<bytes_received4>\t<bytes_received5>\t<bytes_received6>\t<bytes_received7>\t<bytes_received8>\t<bytes_received9>\t<bytes_received10>\t<stamp>\t<average>

PART_PROGRESS_UPDATE¶

Every 10KB of a part download, this message will be sent

PART_PROGRESS_UPDATE <servername>\t<conn_id>\t<subject>\t<part_num>\t<bytes_downloaded>\t<bytes_total>

COLLECTION_ADDED¶

A new collection was added

COLLECTION_ADDED <collection_name>\t<poster>

COLLECTION_REMOVED¶

A collection was removed

COLLECTION_REMOVED <collection_name>

COLLECTION_MODIFIED¶

The poster of a collection was modified

COLLECTION_MODIFIED <collection_name>\t<poster>

FILE_ADDED¶

A file was added to a already existing collection. The <groups> argument is a list of newsgroups, separated by the | character

FILE_ADDED <collection_name>\t<subject>\t<poster>\t<stamp>\t<file_size>\t<status>\t<num_parts>\t<groups>

FILE_REMOVED¶

A file was removed from the download queue

FILE_REMOVED <collection_name>\t<subject>\t<total_size>\t<total_size_remaining>

FILE_DOWNLOAD_STATE_UPDATE¶

The state of a file in the download queue has changed

FILE_DOWNLOAD_STATE_UPDATE <collection_name>\t<subject>\t<num_parts_total>\t<num_parts_done>\t<num_parts_failed>\t<file_size>\t<file_size_remaining>t\t<total_file_size>\t<total_file_size_remaining>

FILE_STATE_CHANGED¶

A file in the download queue was restarted

FILE_STATE_CHANGED <collection_name>\t<subject>\t<real_filename>\t<old_state>\t<new_state>

CONNECTION_CONNECTING¶

A new connection is being made to a usenet server

CONNECTION_CONNECTING <servername>\t<conn_id>

CONNECTION_CONNECTED¶

A connection attempt has succeeded

CONNECTION_CONNECTED <servername>\t<conn_id>\t<welcome_msg>

CONNECTION_DISCONNECT¶

A connection to a usenet server was disconnected

CONNECTION_DISCONNECT <servername>\t<conn_id>\t<disconnect_type>\t<reason>

disconnect_type can be any of the following values:

NORMAL
NO_SUCH_HOST
CONNECTION_REFUSED
TOO_MANY_CONNECTIONS
CONNECT_TIMEOUT
READ_ERROR
READ_TIMEOUT
WRITE_ERROR
IDLE_TIMEOUT
INVALID_MSG
LOGIN_FAILURE
ERROR_SSL_INITIALISE
UNEXPECTED

SCHEDULAR_STATE_CHANGED¶

The state of the schedular has changed

SCHEDULAR_STATE_CHANGED <new_state>\t<reason>

FATAL_ERROR¶

A fatal error has occured

FATAL_ERROR <errmsg>

WARNING_MESSAGE¶

A warning has occured

WARNING_MESSAGE <msg>

DEBUG_MESSAGE¶

A debug message has been emitted

DEBUG_MESSAGE <msg>

TASK_MOVED¶

The position of a task in the download queue has changed

TASK_MOVED <orig_collection_name>\t<subject>\t<new_collection_name>\t<old_position>\t<new_position>

COLLECTION_MOVED¶

The position of a collection in the download queue has changed

COLLECTION_MOVED <collection_name>\t<old_position>\t<new_position>

PAR2_REPAIR_MESSAGE¶

The PAR2 repair component has emit a message which can be shown to the user

PAR2_REPAIR_MESSAGE <message>

ALL_DOWNLOADS_COMPLETED¶

This message is emit when all the items in the download queue are completed

ALL_DOWNLOADS_COMPLETED

UNPACK_PROGRESS_UPDATE¶

The unpack component has made some progress

UNPACK_PROGRESS_UPDATE <collection_name>\t<filename>\t<progress>

UNPACK_MESSAGE_RECEIVED¶

The unpack component has emit a message. This message mostly is the file which is currently extracted

UNPACK_MESSAGE_RECEIVED <collection_name>\t<filename>\t<message>

UNPACK_WORKING_ARCHIVE_CHANGED¶

The unpack component is now unpacking from a new archive file

UNPACK_WORKING_ARCHIVE_CHANGED <collection_name>\t<filename>\t<new archive file>

PAR2_BEGIN_VERIFY¶

A PAR2 verification (and if necessary repair) has just started

PAR2_BEGIN_VERIFY <collection_name>\t<active_par2_filename>

PAR2_LOAD_PROGRESS_UPDATE¶

A file is now being verified

PAR2_LOAD_PROGRESS_UPDATE <collection_name>\t<active_par2_filename>\t<filename>\t<progress>

PAR2_RECOVERY_FILE_LOADED¶

A PAR2 recovery file has been loaded completely

PAR2_RECOVERY_FILE_LOADED <collection_name>\t<active_par2_filename>\t<filename>\t<num_new_packets>\t<num_blocks_found>

PAR2_FILE_LOADED¶

A regular file has been loaded completely

PAR2_FILE_LOADED <collection_name>\t<active_par2_filename>\t<filename>\t<state>\t<num_blocks_found>\t<num_blocks_expected>

The field state can be one of the following values:

MISSING
FOUND
DAMAGED
NO_NEW_BLOCKS_FOUND

PAR2_REPAIR_PROGRESS_UPDATE¶

A PAR2 repair is now going on

PAR2_REPAIR_PROGRESS_UPDATE <collection_name>\t<active_par2_filename>\t<progress>

PAR2_REPAIR_FAILURE¶

The PAR2 repair has failed because there aren't enough recovery blocks available

PAR2_REPAIR_FAILURE <collection_name>\t<active_par2_filename>\t<num_blocks_more_required>

PAR2_REPAIR_SUCCESS¶

The PAR2 repair has completed successfully

PAR2_REPAIR_SUCCESS <collection_name>\t<active_par2_filename>

PAR2_NO_REPAIR_REQUIRED¶

All the files in the PAR2 set have been verified and no repair is necessary

PAR2_NO_REPAIR_REQUIRED <collection_name>\t<active_par2_filename>

Also available in: HTML TXT

NNTPGrab

Wiki

Protocol¶

MODE COMMAND¶

CONFIG_GET_AVAIL_SERVERS¶

CONFIG_GET_SERVER_INFO¶

CONFIG_ADD_SERVER¶

CONFIG_DEL_SERVER¶

CONFIG_EDIT_SERVER¶

CONFIG_GET_OPTS¶

CONFIG_SET_OPTS¶

CONFIG_GET_FOLDER_LISTING¶

SCHEDULAR_START¶

SCHEDULAR_STOP¶

SCHEDULAR_GET_STATE¶

SCHEDULAR_ADD_TASK_TO_QUEUE¶

SCHEDULAR_DEL_TASK_FROM_QUEUE¶

SCHEDULAR_RESTART_TASK¶

SCHEDULAR_SAVE_QUEUE¶

SCHEDULAR_FOREACH_TASK¶

SCHEDULAR_MOVE_TASK¶

SCHEDULAR_MOVE_COLLECTION¶

MODE LISTENER¶

CONFIG_CHANGED¶

PART_DOWNLOAD_START¶

PART_DONE¶

PART_FAILED¶

TRAFFIC_MONITOR_UPDATE¶

PART_PROGRESS_UPDATE¶

COLLECTION_ADDED¶

COLLECTION_REMOVED¶

COLLECTION_MODIFIED¶

FILE_ADDED¶

FILE_REMOVED¶

FILE_DOWNLOAD_STATE_UPDATE¶

FILE_STATE_CHANGED¶

CONNECTION_CONNECTING¶

CONNECTION_CONNECTED¶

CONNECTION_DISCONNECT¶

SCHEDULAR_STATE_CHANGED¶

FATAL_ERROR¶

WARNING_MESSAGE¶

DEBUG_MESSAGE¶

TASK_MOVED¶

COLLECTION_MOVED¶

PAR2_REPAIR_MESSAGE¶

ALL_DOWNLOADS_COMPLETED¶

UNPACK_PROGRESS_UPDATE¶

UNPACK_MESSAGE_RECEIVED¶

UNPACK_WORKING_ARCHIVE_CHANGED¶

PAR2_BEGIN_VERIFY¶

PAR2_LOAD_PROGRESS_UPDATE¶

PAR2_RECOVERY_FILE_LOADED¶

PAR2_FILE_LOADED¶

PAR2_REPAIR_PROGRESS_UPDATE¶

PAR2_REPAIR_FAILURE¶

PAR2_REPAIR_SUCCESS¶

PAR2_NO_REPAIR_REQUIRED¶