Group call media subscription

This document describes the integration method for requesting group call media subscription and receiving media data on the Planet Cloud platform.

Overview

To request group call media subscription, you need to implement a subscription client (WebSocket client). Group call media subscription works as follows:

The subscription client requests authentication from the gateway server of Planet Cloud to request group call media subscription.
The gateway server processes authentication, generates a subscription token, and sends the token to the client.
The subscription client requests group call media subscription to Planet Cloud's subscription server.
The subscription server sends group call media to the subscription client using WebSocket connection established by the subscription client's request.

The following figure shows the integration structure for group call media subscription.

Integration structure diagram for group call media subscription

Note

The app server can also be implemented to include the subscription client functionality. In this case, the app server will directly interact with the gateway server and the subscription server.

This document assumes that the app server and the subscription client are implemented separately and provides description accordingly.

API call sequence

The following diagram shows the sequence of API calls for requesting group call media subscription and receiving media data.

Group call media subscription API call sequence

Gateway server API specification

This section describes an API to get a subscription token from the gateway server for authentication.

Access information

Environment	Base URL
Evaluation	`https://vpnx-stn-api.line-apps-rc.com`

For more information on authentication, refer to Basic protocol.

API description

Gets a subscription token for connecting to subscription server.

The token validity period is as follows:

Environment	Token validity period
Evaluation	1 day
Real	1 day

Method and endpoint

Method: GET

Endpoint

/tas/v1/gcall/{serviceId}/{roomId}/subs/token

Authorization: Basic {your credential}
Content-Type: text/plain

Path parameters

Parameter	Description
`serviceId`	Service ID
`roomId`	Room ID

Query

N/A

Response

Subscription token

Example

curl --location --request GET 'https://vpnx-stn-api.line-apps-rc.com/tas/v1/gcall/planetkit-inhouse/1024/subs/token' \
--header 'Authorization: Basic em45eEkxRnROYlZwYlNXWFo1U2ZCWmhucF85dHFGaXBod2h0TjdNLTdTWHlFbllQalB2dUVKRXBDdkx5OU1tRWlHUEI2clpITjdvSVA3UXVWSE5sN3BmeHlObkxlRld0dkFCT2p4MkpCbXBROFZPVXZmMU11dVZ3QWtfdGU2cUh0OFVpV250TzZaZzZqdWIteWdtRnNBOjc4YTQwN2JjOGM0NjljZjE3MjI2MWVmY2E0MWJmODBlYzY5ODM1ZTY1MDk0YTJjZjc2YzAxYWE5N2U5Y2Q1MWU=' \
--header 'Content-Type: text/plain'

eea3091e49674a13a3091e49671a13c7

Possible error codes

Possible HTTP status codes are as follows. For more information, see Error handling.

400, 401, 403, 404, 429, 500, 503

Subscription server API specification

This section describes an API that handles subscription of media data for the group call.

Access information

Environment	Base URL
Evaluation	`wss://voipnx-rhea.line-apps-rc.com`

API description

Requests a WebSocket connection by using the session_id as a path parameter.

If no valid message is received within 1 second after establishing the WebSocket connection, the connection is terminated.
When the subscription client receives a PING (opcode 0x09) message from the server, the subscription client must immediately respond with a PONG (opcode 0x0a) message to maintain the connection.

Warning

Subscription servers do not support high availability. If the WebSocket connection is disconnected due to a network or server issue during subscription, the subscription client must re-establish the WebSocket connection and request the subscription again.

Method and endpoint

Method: GET
Endpoint
```
/subs_gcall/{session-id}
```

Control message

Control messages are delivered using text opcode of WebSocket. Control message have the following characteristics:

Uses JSON format.
Consists of header and body.

Header

Field	Data type	Description
`msg_id`	Number	- `SubscribeGroupcallMediaReq` = 1 - `SubscribeGroupcallMediaRsp` = 2 - `UnsubscribeGroupcallMediaReq` = 3 - `UnsubscribeGroupcallMediaRsp` = 4 - `AbortGroupcallMediaSubscriptionRpt` = 5 - `ChangeGroupcallMediaSubscriptionReq` = 6 - `ChangeGroupcallMediaSubscriptionRsp` = 7 - `UpdateGroupcallMediaContributorsRpt` = 8 - `NotifyGroupcallUserStatusRpt` = 9
`sess_id`	String	Session ID generated by the subscription client

`SubscribeGroupcallMediaReq` (`msg_id`: 1)

This message requests Planet Cloud to subscribe to media data for the group call.

Before making a request with this message, you must obtain subsToken by requesting authentication from the Planet Cloud gateway server.

Message body

Field	Data type	Description	Note
`service_id`	String	Service ID
`room_id`	String	Room ID
`credential`	String	Base64-encoded string
`media_type`	Array (String elements)	List of media types, where `A` indicates audio and `V` indicates video	Supported values are as follows: - `A` - `A, V` Video only mode is not supported.
`stream_type`	String	Type of the stream you want to receive - `MIX`: Provides a mixed stream for the requested target - `MULTI`: Provides a separate stream by ID for the requested target	Supported values depending on `media_type` are as follows: - `A`: `MIX`, `MULTI` - `A, V`: `MIX`
`target_type`	String	Subscription target type - `MAIN_ROOM`: Subscribes to the main room - `SUBGROUP`: Subscribes to a subgroup - `TRANSLATE`: Subscribes to translation room and subgroup	This field is valid only when `stream_type` is `MIX`.
`subgroup_name`	String	Name of the subgroup you want to subscribe to	This field is valid only when `target_type` is `SUBGROUP`.
`target_user_ids`	Array (String elements)	List of IDs of users you want to subscribe to	If there is no `target_user_ids` field or the field contains no items, all participants are the target of subscription. If `target_type` is `MAIN_ROOM`, the participant in the main room is selected, and if `target_type` is `SUBGROUP`, the participant in the subgroup is selected. If `target_type` is `TRANSLATE`, `target_user_ids` is not allowed. The maximum number of elements that can be set in `target_user_ids` is 16.
`layout_type`	String	Layout type for video mixing. For more information, see Layout types.	This field is valid only when `media_type` contains `V` (Video).
`max_display_num`	Number	- Grid view: 1–16 - Focus view: 1–8 - Presentation view: 1–8	The maximum number of participants that can be displayed on the video depends on the layout type, and in the case of grid view, the maximum is 16.
`auto_screenshare`	Bool	Whether to automatically switch to presentation view when screen share starts
`no_target_users_timeout`	Number	When the `target_user_ids` parameter is set, the maximum time to keep a subscription session when there is no target user in the room (in seconds)	If the subscription client does not set this value, the subscription session is terminated according to the system setting.

Note for the case when target_type is SUBGROUP

In one subscription session, only one subgroup can be subscribed to.

To subscribe to another subgroup created in the main room, you need to establish a new subscription session and request subscription to the subgroup.

Layout types

Values available for the layout_type field are as follows:

Value of `layout_type`	Availability of `target_user_ids`	Description
`TALKER_GRID_VIEW`	N	A grid view where the new talker has priority. When a new talker needs to enter the layout but there is no space, the participant who did not talk for the longest time is excluded.
	Y	If `target_user_ids` is specified, only those users are included in the layout.
`ORDERED_GRID_VIEW`	N	Not available
	Y	A grid view where the layout order is determined by the order of the `target_user_ids` elements.
`TALKER_FOCUS_VIEW`	N	A focus view where the new talker has priority. The new talker is displayed in the full-view area and the previous talker goes to the first thumbnail area. If the thumbnail view is full, the participant who did not talk for the longest time is excluded.
	Y	If `target_user_ids` is specified, only those users are included in the layout. The new talker is displayed in the full-view area and the previous talker goes to the first thumbnail area.
`ORDERED_FOCUS_VIEW`	N	Not available
	Y	A focus view where the layout order is determined by the order of the `target_user_ids` elements. The first user in `target_user_ids` is displayed in the full-view area and the previous talker goes to the first thumbnail area.

Example

{
  "header": {
    "msg_id": 1,
    "sess_id": "sid343213"
  },
  "body": {
    "service_id": "planet",
    "room_id": "RID_xlkjds",
    "credential": "YW55IGNhcm5hbCBwbGVhcw==",
    "media_type": ["A"],
    "stream_type": "MULTI",
    "target_type": "MAIN_ROOM"
  }
}

{
  "header": {
    "msg_id": 1,
    "sess_id": "sid343213"
  },
  "body": {
    "service_id": "planet",
    "room_id": "RID_xlkjds",
    "credential": "YW55IGNhcm5hbCBwbGVhcw==",
    "media_type": ["A", "V"],
    "stream_type": "MIX",
    "target_type": "MAIN_ROOM",
    "layout_type": "TALKER_GRID_VIEW"
  }
}

Calculating credential of the subscription request

When requesting group call media subscription, the following calculated value must be included in the request body as the value of the credential field.

credential value: A value calculated using the SHA-256 hash algorithm with the subsToken value generated from the Planet gateway server and group call related information as input values.

The subscription server uses the credential value of the request body to check whether the subscription request is authorized.

Input values

serviceID
roomID
subsToken

Implementation example pseudo code

credential calculation
#define AUTH_DELIM "::"
#define SHA256_HASHLEN 32
typedef unsigned char SHA256_HASH[SHA256_HASHLEN];

SHA256_CTX ctx;
SHA256_HASH SHA256_HA;
SHA256_Init(&ctx);
SHA256_Update(&ctx, svc_id, strlen(svc_id));
SHA256_Update(&ctx, AUTH_DELIM, strlen(AUTH_DELIM));
SHA256_Update(&ctx, room_id, strlen(room_id));
SHA256_Update(&ctx, AUTH_DELIM, strlen(AUTH_DELIM));
SHA256_Update(&ctx, subs_token, strlen(subs_token));
SHA256_Final(SHA256_HA, &ctx)

Java example code

import java.nio.charset.StandardCharsets;

import org.apache.commons.codec.binary.Base64;

import com.google.common.hash.Hashing;

public class SubsUtils {
    public static final String del = "::";

    public static String makeCredential(String svcId, String roomId, String subsToken) {
        byte[] hash = Hashing.sha256()
                             .newHasher()
                             .putString(svcId, StandardCharsets.UTF_8)
                             .putString(del, StandardCharsets.UTF_8)
                             .putString(roomId, StandardCharsets.UTF_8)
                             .putString(del, StandardCharsets.UTF_8)
                             .putString(subsToken, StandardCharsets.UTF_8)
                             .hash()
                             .asBytes();
        String base64EncodedCredential = new String(Base64.encodeBase64(hash));
        return base64EncodedCredential;
    }
}

`SubscribeGroupcallMediaRsp` (`msg_id`: 2)

This is a response message to a group call media subscription request. It contains subs_id created by the subscription server and details of transmittable media format. When media data is transmitted through binary opcode of WebSocket, it is delivered in Media data message format.

Message body

Field	Data type	Description	Note
`result`	String	Result of the API - `success` - `fail`
`subs_id`	Number	Subscription ID generated by the subscription server	A value exists when `result` is `success`.
`audio_payload_info`	Array (Audio PayloadInfo Object elements)	Detailed information about the `pt` (payload type) value representing the audio packet	A value exists when `result` is `success`.
`video_payload_info`	Array (Video PayloadInfo Object elements)	Detailed information about the `pt` value representing the video packet	A value exists when `result` is `success`.
`error_code`	Number	Error code. For more information, see Error codes.	A value exists when `result` is `fail`.
`error_desc`	String	Detailed error description	A value exists when `result` is `fail`.

Audio PayloadInfo Object

Name	Type	Description	Example
`pt`	Number	Payload type ID	`120`
`codec`	String	Audio codec name	`"opus"`
`clock_rate`	Number	Clock rate	`48000`
`channels`	Number	Audio channel count	`1`
`ptime`	Number	Packetization interval time (milliseconds)	`20`

Video PayloadInfo Object

Name	Type	Description	Example
`pt`	Number	Payload type ID	`121`
`codec`	String	Video codec name	`"H264"`
`clock_rate`	Number	Clock rate	`90000`

Example

{
  "header": {
    "msg_id": 2,
    "sess_id": "sid343213"
  },
  "body": {
    "result": "success",
    "subs_id": 2394872139847981,
    "audio_payload_info": [
      {
        "pt": 120,
        "codec": "opus",
        "clock_rate": 48000,
        "channels": 1,
        "ptime": 20
      }
    ]
  }
}

{
  "header": {
    "msg_id": 2,
    "sess_id": "sid343213"
  },
  "body": {
    "result": "success",
    "subs_id": 2394872139847981,
    "audio_payload_info": [
      {
        "pt": 120,
        "codec": "opus",
        "clock_rate": 48000,
        "channels": 1,
        "ptime": 20
      }
    ],
    "video_payload_info": [
      {
        "pt": 121,
        "codec": "H264",
        "clock_rate": 90000
      }
    ]
  }
}

{
  "header": {
    "msg_id": 2,
    "sess_id": "sid343213"
  },
  "body": {
    "result": "fail",
    "error_code": 100,
    "error_desc": "Authentication failed"
  }
}

`UnsubscribeGroupcallMediaReq` (`msg_id`: 3)

This message requests stop of group call media subscription.

Message body

Field	Data type	Description
`service_id`	String	Service ID
`room_id`	String	Room ID
`subs_id`	Number	Subscription ID generated by the subscription server

Example

{
  "header": {
    "msg_id": 3,
    "sess_id": "sid343213"
  },
  "body": {
    "service_id": "planet",
    "room_id": "RID_xlkjds",
    "subs_id": 2394872139847981
  }
}

`UnsubscribeGroupcallMediaRsp` (`msg_id`: 4)

This is a response message to a request for stopping group call media subscription.

Message body

Field	Data type	Description	Note
`result`	String	Result of the API - `success` - `fail`
`error_code`	Number	Error code. For more information, see Error codes.	A value exists when `result` is `fail`.
`error_desc`	String	Detailed error description	A value exists when `result` is `fail`.

Example

{
  "header": {
    "msg_id": 4,
    "sess_id": "sid343213"
  },
  "body": {
    "result": "success",
  }
}

`AbortGroupcallMediaSubscriptionRpt` (`msg_id`: 5)

This is a message to notify the subscription client when subscription is interrupted due to an event occurring in the Planet Cloud GroupCall server.

Message body

Field	Data type	Description
`error_code`	Number	Error code. For more information, see Error codes.
`error_desc`	String	Detailed error description

Example

{
  "header": {
    "msg_id": 5,
    "sess_id": "sid343213"
  },
  "body": {
    "error_code": 2,
    "error_desc": "Group call has been ended"
  }
}

`ChangeGroupcallMediaSubscriptionReq` (`msg_id`: 6)

This message is used to change media subscription information. You can update the following item.

Video mixing layout type

Message body

Field	Data type	Description	Note
`service_id`	String	Service ID
`room_id`	String	Room ID
`subs_id`	Number	Subscription ID generated by the subscription server
`layout_type`	String	Layout type for video mixing	This field is valid only when `media_type` contains `V` (Video).

Example

{
  "header": {
    "msg_id": 6,
    "sess_id": "sid343213"
  },
  "body": {
    "service_id": "planet",
    "room_id": "RID_xlkjds",
    "subs_id": 2394872139847981,
    "layout_type": "TALKER_FOCUS_VIEW"
  }
}

`ChangeGroupcallMediaSubscriptionRsp` (`msg_id`: 7)

This message returns the result of changing subscription information.

Message body

Field	Data type	Description	Note
`result`	String	Result of the API - `success` - `fail`
`error_code`	Number	Error code. For more information, see Error codes.	A value exists when `result` is `fail`.
`error_desc`	String	Detailed error description	A value exists when `result` is `fail`.

Example

{
  "header": {
    "msg_id": 7,
    "sess_id": "sid343213"
  },
  "body": {
    "result": "success",
  }
}

`UpdateGroupcallMediaContributorsRpt` (`msg_id`: 8)

When the stream_type is MIX and there is a change in the user_id list included in the mixed media, the updated user_id list is delivered to the subscription client through this message.

Message body

Field	Data type	Description	Example
`pt`	Number	Payload type ID	`120` (audio) or `121` (video)
`ts`	Number	RTP timestamp
`seq`	Number	RTP sequence number
`user_ids`	Array (String elements)	List of `user_id`

Example

{
  "header": {
    "msg_id": 8,
    "sess_id": "sid343213"
  },
  "body": {
       "pt": 120,
       "ts": 3475364964,
       "seq": 30968,
       "user_ids": ["user1", "user2", "user3"]
   }
}

`NotifyGroupcallUserStatusRpt` (`msg_id`: 9)

When the target_user_ids parameter is set when requesting subscription (SubscribeGroupcallMediaReq), when a user included in this parameter enters or leaves the room, the users' status information is delivered to the subscription client through this message.

Message body

Field	Data type	Description
`enter`	Array (String elements)	List of IDs of users who entered the room
`leave`	Array (String elements)	List of IDs of users who left the room

Example

{
  "header": {
    "msg_id": 9,
    "sess_id": "sid343213"
  },
  "body": {
         "enter": ["user1", "user2", "user3"],
         "leave": ["user4", "user5"]
    }
}

Error codes

Error code	Reason	Note
0	`UNDEFINED`
1	`NO_ERROR`
2	`GROUPCALL_END`
100	`AUTHENTICATION_FAIL`
101	`ROOM_NOT_FOUND`
102	`SUBSCRIPTION_NOT_SUPPORT`
103	`INVALID_PARAMETER`
104	`SUBSCRIPTION_NOT_FOUND`
105	`INVALID_STATE`
106	`INVALID_MSG_FORMAT`
107	`SUBS_CAPA_FULL`
200	`INTERNAL_SERVER_ERROR`
201	`WS_DISCONNECTED_BY_PEER`
202	`WS_PING_PONG_FAIL`
203	`WS_INITIAL_REQUEST_TIMEOUT`
1000	`UNKNOWN`

Media data message

Media data messages are delivered using binary opcode of WebSocket, and are delivered in two types as described below, depending on the requested stream_type.

Media data message format

The format of media data message is as follows:

Seq	Property	Size	Description
1	`type`	1 byte	Type of data 1: RTP (Mixed audio or video) 2: RTP with `user_id`
2	`data`	Variable length

Type 1: Mixed audio or video RTP

Target stream_type: MIX
Target media_type: A (audio) or A, V (audio and video)
The mixed audio and video are transmitted through a connected WebSocket session in RTP formatted data.
Audio and video are distinguished through the pt (payload type) of the transmitted RTP header.

Media data messages of type 1 have the following format:

Seq	Property	Size
1	Audio or video RTP	Variable length

Type 2: Audio RTP of a specific user

Target stream_type: MULTI
Target media_type: A (audio)
The audio data of each user participating in the group call is delivered to the subscription client in the form of user_id information (header) and RTP.

Media data messages of type 2 have the following format:

Seq	Property	Size
1	`user_id_length`	1 byte
2	`user_id`	Variable length
3	Audio RTP	Variable length

Warning

The RTP context (timestamp, sequence) of a specific user (group call participant) can be restarted in the following cases:

When the user stops talking for a certain amount of time and then starts talking again
When the user joins (or re-joins) the call and starts talking

Overview​

API call sequence​

Gateway server API specification​

Access information​

API description​

Method and endpoint​

Header​

Path parameters​

Query​

Response​

Example​

Possible error codes​

Subscription server API specification​

Access information​

API description​

Method and endpoint​

Control message​

Header​

SubscribeGroupcallMediaReq (msg_id: 1)​

Message body​

Layout types​

Example​

Calculating credential of the subscription request​

Input values​

Implementation example pseudo code​

Java example code​

SubscribeGroupcallMediaRsp (msg_id: 2)​

Message body​

Audio PayloadInfo Object​

Video PayloadInfo Object​

Example​

UnsubscribeGroupcallMediaReq (msg_id: 3)​

Message body​

Example​

UnsubscribeGroupcallMediaRsp (msg_id: 4)​

Message body​

Example​

AbortGroupcallMediaSubscriptionRpt (msg_id: 5)​

Message body​

Example​

ChangeGroupcallMediaSubscriptionReq (msg_id: 6)​

Message body​

Example​

ChangeGroupcallMediaSubscriptionRsp (msg_id: 7)​

Message body​

Example​

UpdateGroupcallMediaContributorsRpt (msg_id: 8)​

Message body​

Example​

NotifyGroupcallUserStatusRpt (msg_id: 9)​

Message body​

Example​

Error codes​

Media data message​

Media data message format​

Type 1: Mixed audio or video RTP​

Type 2: Audio RTP of a specific user​

Overview

API call sequence

Gateway server API specification

Access information

API description

Method and endpoint

Header

Path parameters

Query

Response

Example

Possible error codes

Subscription server API specification

Access information

API description

Method and endpoint

Control message

Header

`SubscribeGroupcallMediaReq` (`msg_id`: 1)

Message body

Layout types

Example

Calculating credential of the subscription request

Input values

Implementation example pseudo code

Java example code

`SubscribeGroupcallMediaRsp` (`msg_id`: 2)

Message body

Audio PayloadInfo Object

Video PayloadInfo Object

Example

`UnsubscribeGroupcallMediaReq` (`msg_id`: 3)

Message body

Example

`UnsubscribeGroupcallMediaRsp` (`msg_id`: 4)

Message body

Example

`AbortGroupcallMediaSubscriptionRpt` (`msg_id`: 5)

Message body

Example

`ChangeGroupcallMediaSubscriptionReq` (`msg_id`: 6)

Message body

Example

`ChangeGroupcallMediaSubscriptionRsp` (`msg_id`: 7)

Message body

Example

`UpdateGroupcallMediaContributorsRpt` (`msg_id`: 8)

Message body

Example

`NotifyGroupcallUserStatusRpt` (`msg_id`: 9)

Message body

Example

Error codes

Media data message

Media data message format

Type 1: Mixed audio or video RTP

Type 2: Audio RTP of a specific user