Source code explanation for Silverlight VideoChat Example

Download:

05_ChatExample.zip

This page is entitled to be an explanation page for the source code of Silverlight VideoChat Example to provide an overall view. Please check the source code parts and their explanation below this page.

Server-side implementation

The server is responsible for receiving the connection requests from the clients and setup the data communication between the connected clients. The server is created by the Ozeki VoIP SIP SDK, this way it is ideal for serving both Silverlight and Flash clients by using the same source code. The clients connect to the server with a nickName parameter. The server will „decide” based on this parameter that the given client with the given nickName is allowed to login the server or not. The successfully connected clients are stored in a Dictionary collection for further use.

The server is a console application. The ChatGateway class describes the server functionality toward the clients. This class will use and extend the basic server implementation and functionality (client connection handling, data forwarding services, mechanisms) that is provided by the Ozeki VoIP SIP SDK. The object of ChatGateway class is created by the Main(string[] args) method (that is the entry point of the application) in the following way shown in Code 1.

static void Main(string[] args)
        {
            MediaGatewayConfig mediaConfig = new MediaGatewayConfig();
            mediaConfig.AddConfigElement(new SilverlightConfig();
            //mediaConfig.AddConfigElement(new FlashConfig());
            var mediaGateway = new ChatGateway(mediaConfig);
            mediaGateway.Start();
            Console.WriteLine("Video chat service Started!");
            Console.WriteLine("Press enter to shut down service and exit.\n\n");
            Console.ReadLine();
        }
 

Code 1 - The main method for the server application

The ChatGateway derives from the MediaGateway, this way its constructor can be called in two ways: without a parameter and with a MediaGatewayConfig parameter. In the first case, the constructor expects the configuration from the appConfig.xml file of the application, while in the second case, it expects the configuration from the parameter.

In this example, the second option is used, so a MediaGatewayConfig object is created. Then a SilverlightConfig is added to the created object. The SilverlightConfig is for serving the clients. In order to make the server to be able to server Flash clients, as well, the FlashConfig configuration item can also added to it, but it may cause some problem in proper working if you use both client types at the same time.

It can be seen that it is added without a parameter, because the constructor has default parameters and the default values of these parameters are ideal for this example. The SilverlightConfig object can also be added in this way, but this example is for demonstrating both options.

After creating the configuration object, the ChatGateway object is created. It is responsible for the implementation of the server’s functionality. Then the server’s services are started with the Start() method that is invoked on the object.

Now let’s see what functionalities are provided by the ChatGateway for the server. It stores the nicknames of logged in clients and the client reference that belongs to them. It makes the connection and logout and forwards data from one client to the other. As it was already mentioned, the basic server implementation is provided by the Ozeki VoIP SIP SDK. The Ozeki VoIP SIP SDK provides this basic server implementation via its MediaGateway object. This way, the ChatGateway class will also be derived from it (Code 2).

class ChatGateway : MediaGateway

Code 2 - The ChatGateway is defined as a subclass of the MediaGateway class

Connected clients are stored in the following way shown in Code 3:

private Dictionary<string,IClient> chatClients;

Code 3 - The clients are stored in a Dictionary collection on the server's side

The dictionary key is the string that includes the given client’s nickname and the associated value is the client reference via which it is possible to connect to the client.

The ’OnClientConnect’ method is responsible for handling client connections (Code 4).

public override void OnClientConnect(IClient client,  object[] parameters)
        {
            string nickname = parameters[0] as string;

            if (String.IsNullOrEmpty(nickname))
                return;
            Console.WriteLine("New client '{0}' is trying connect.", nickname);

            if (!chatClients.ContainsKey(nickname))
            {
                chatClients.Add(nickname,client);
                Console.WriteLine("Client '{0}' connected successfully.", nickname);
                ConnectedClientChanged(client);
                base.OnClientConnect(client, parameters);
            }
            else
            {
                Console.WriteLine("Nickname: '{0}' already has been used.",nickname);
            }
        }

Code 4 - Client connection handling on the server-side is done with the OnClientConnect method

It can be seen that one client can receive a reference and an object[] parameter. In this example, only one extra parameter is used. This extra parameter is the client’s nickname. When a new user is connected, its nickname is added to the collection of the previously connected users and a notification is also sent to the already connected users about the change.

public override void OnClientDisconnect(IClient client)
        {
            if (chatClients.ContainsValue(client))
            {
                foreach (KeyValuePair<string, IClient> keyValuePair in chatClients)
                {
                    if (keyValuePair.Value==client)
                    {
                        Console.WriteLine("'{0}' client disconnected.", keyValuePair.Key);
                        chatClients.Remove(keyValuePair.Key);
                        break;
                    }
                }
            }
            ConnectedClientChanged(client);
            base.OnClientDisconnect(client);
        }

Code 5 - The clients' disconnection is handled with this method

When a client disconnects, it will be removed from the connected client’s list, and the connected clients will be notified about the change (Code 5). It is necessary to notify new and already connected clients when a client connects or disconnects.

  public void GetConnectedClients(IClient client, string requestOwnernickName)
        {
            List<string> users;

            users = chatClients.Keys.ToList();

            if (users.Contains(requestOwnernickName))
                users.Remove(requestOwnernickName);

            client.InvokeMethod("ConnectedClientsReceived", new object[] {users.ToArray()});
        }

Code 6 The GetConnectedClients method

As opposed to the two previous methods, GetConnectedClients (Code 6) method (and the further methods below this page) is not part of the basic tools of the VoIP SIP SDK. This way, these methods will be invoked in a different way. These methods are invoked via the invokeOnConnection method of the MediaConnection class that represents the server on the client side. So this method is invoked by the Flash (in that case it is a function) or the Silverlight client after connecting in order to get know with which clients it can communicate on the server. When a client is connected or disconnected, the already connected clients needs to be notified about the change. This task is implemented by ConnectedClientChanged (Code 7) method.

private void ConnectedClientChanged(IClient requestClient)
        {
            try
            {
                foreach (var client in chatClients)
                {
                    if (client.Value == requestClient)
                        continue;
                    client.Value.InvokeMethod("ConnectedClientsReceived", new object[] { chatClients.Keys.ToList().ToArray() });
                }
            }
            catch (Exception)
            {}
        }

Code 7 - This method notifies the clients about a client connection or disconnection

Here the ConnectedClientsReceived methods of the connected clients are invoked one by one with the actual client list. The server is also responsible for implementing data communication between the connected clients, so one client will be able to send text, audio and video data to the other client. The most simple is to send text message. It is done as seen in Code 8.

public void SendText(IClient client, string owner, string target, string msg)
        {
            if (chatClients.ContainsKey(target))
            {
                IClient cl;
                chatClients.TryGetValue(target,out cl);
                cl.InvokeMethod("ReceiveMessage", owner, msg);
            }
        }

Code 8 - Simple text message sending from one client to another

Here the server checks if the given client to which it wishes to send data is still connected or not. If it is still connected, then the server invokes the given client’s ReceiveMessage method.

Audio and video functionality works in the following way between two clients. The initiating client sends an audio/video chat request to the target client. The target client can accept or reject this request. The initiating client will be notified about the target client’s choice (if it accepts or rejects the request) in a response message. In case the request is accepted, the two clients start sampling their input media and send it to the server. The server will forward the media to the other client. This example demonstrates only audio chat because the logical implementation of audio and video chat can be done in the same way.

  public void SendAudioRequest(IClient client, string owner, string target, bool isEnable)
        {
            if (chatClients.ContainsKey(target))
            {
                IClient cl;
                chatClients.TryGetValue(target, out cl);
                cl.InvokeMethod("AudioRequestReceived", owner, isEnable);
            }
        }

Code 9 - The clients initiate audio chat with sending audio request to another client

It can be seen in code 9 that audio requests are sent similarly to sending text messages. It is true for the response messages, as well. Code 10 shows the response method for the audio request that is for accepting or rejecting of an audio chat request from another client.

public void SendAudioResponse(IClient client, string owner, string target, bool response)
        {
            if (chatClients.ContainsKey(target))
            {
                IClient cl;
                chatClients.TryGetValue(target, out cl);
                cl.InvokeMethod("AudioResponseReceived", owner, response);
            }
        }

Code 10 - The client can accept or reject an audio chat request by calling this method

The above mentioned methods are for setting up audio/video chat communication. The actual forwarding of media data will be done via the built-in mechanism of Ozeki VoIP SIP SDK.

Silverlight client

This program is a simple chat application with text/audio/video functionality. On startup it requires a nickname with which it logins to the server mentioned above. It can communicate with other logged in clients via this server.

The program starts with an opening Silverlight ChildWindow window. It is responsible for asking for the nickname and connecting for the server with this given nickname. MediaConnection object is used to reach the server that has been created with the help of Ozeki VoIP SIP SDK.

void CWindowConnection_Loaded(object sender, RoutedEventArgs e)
        {
           connection = new MediaConnection("127.0.0.1:4502/SilverlightMediaGateway");
           connection.ConnectionStateChanged += new EventHandler<GenericEventArgs<ConnectionState>>(connection_ConnectionStateChanged);

        }
        

Code 11 - The basic initialization of the connection

The instance of the MediaConnection object has been created with the IP address of the server to which it will connect with connection.Connect(txtNickName.Text); order (Code 11). The response for this order will be received in the event handler of ConnectionStateChanged event (Code 12).

   void connection_ConnectionStateChanged(object sender, GenericEventArgs<ConnectionState> e)
        {
            switch (e.Item)
            {
                case ConnectionState.Success:
                    lblStatus.Text = "Online";
                    if (ConnectedSuccessfully!= null)
                    {
                        ConnectedSuccessfully(this, new GenericEventArgs<MediaConnection>(connection));
                    }
                    this.DialogResult = true;
                    break;
.
.
.
            }
        }
        

Code 12 - The EventHandler method for the connection state changing

The fact of successful connection is indicated to the main window of the Silverlight application via a ConnectedSuccessFully event (Code 13).

void conWindow_ConnectedSuccessfully(object sender, GenericEventArgs<MediaConnection> e)
        {
            connection = e.Item;
            rectOffline.Visibility = System.Windows.Visibility.Collapsed;
            lblNickName.Text = conWindow.txtNickName.Text;
            txtChatLog.Text += "Connected successfuly.\n";
            connection.Client = this;
            connection.InvokeOnConnection("GetConnectedClients",lblNickName.Text);
            streamSender=new MediaStreamSender(connection);
            streamSender.StreamStateChanged += new EventHandler<GenericEventArgs<StreamState>>(streamSender_StreamStateChanged);
            streamSender.Publish(lblNickName.Text);
        }
        

Code 13 - The successful connection will invoke a ConnectedSuccessfully event

Where the reference that is needed to reach the server is stored, and the object that includes the client methods invited from the server-side is set for the given reference. connection.Client = this;

In order to get a list of the clients that connected to the server after login, the GetConnectedClients method of the server is invited. This method will recall a method of the client with the connected clients shown in Code 14.

  public void ConnectedClientsReceived(string[] connectedUsers)
        {
            this.connectedUsers.Clear();
            foreach (string user in connectedUsers)
            {
                if (user==lblNickName.Text)
                    continue;
                this.connectedUsers.Add(user);
            }
        }
        

Code 14 - The client gets the list of connected clients via this method from the server

Furthermore, a MediaStreamSender object is created after connection and then it is published with the nickname that has been specified on login. This MediaStreamSender object is responsible for encoding data receiving from the microphone and the camera, and then it sends these data to the server.

  streamSender=new MediaStreamSender(connection);
            streamSender.StreamStateChanged += new EventHandler<GenericEventArgs<StreamState>>(streamSender_StreamStateChanged);
            streamSender.Publish(lblNickName.Text);
            

Code 15 - The basic initiation steps for the MediaSender object

After these steps, the application is ready for communication.

Sending and receiving messages

Text messages are sent to another client by the btnSend_Click method that has been discussed at the server side implementation (Code 16).

connection.InvokeOnConnection("SendText", lblNickName.Text, listConnectedUsers.SelectedItem, txtMsgInput.Text);

Code 16 - The text message sending works directly between two clients

Messages are received via the MainPage.cs - ReceiveMessage method (Code 17) that is invited by the server side.

   public void ReceiveMessage(string owner, string message)
        {
            txtChatLog.Text += String.Format("{0}: {1}\n", owner, message);
        }
        

Code 17 This method is responsible for the message receiving

Where the received message is simply appended to the data source of a listBox.

Sending/Receiving audio/video data

Transferring these media data is more complex than transferring text data. That is why, Ozeki VoIP SIP SDK includes the support that allows to implement these functionalities in a simpler way. The basic idea is the following: the connected clients are able to publish MediaStreamSender objects with a unique ID on the server. Other clients are able to play these objects via a MediaStreamReceiver object, since the Play function is called on this object with the published ID of the MediaStreamSender object.

When a client wants to initiate an audio/video chat with another client, first it will send an audio/video request to the server (Code 18). The server then forwards these requests for the appropriate media to the target client. Then a response is also sent back via the server.

if (listConnectedUsers.SelectedItem != null && (CaptureDeviceConfiguration.AllowedDeviceAccess || CaptureDeviceConfiguration.RequestDeviceAccess()))
            {

                audioIsEnable = !audioIsEnable;
                txtChatLog.Text += "Please wait for other person response.\n";
                connection.InvokeOnConnection("SendAudioRequest", lblNickName.Text, listConnectedUsers.SelectedItem, audioIsEnable);
            }
            

Code 18 - Audio request sending

Based on the above mentioned the establishment of the communication is made as follows: In order to get access to the camera and the microphone, Silverlight requires that the user to allow the use of their microphone/camera via "user interaction". If the user allows the use of the microphone and the camera successfully, then the request related to the media is sent to the remote end. The response arrives in AudioResponseReceived method (Code 19).

public void AudioResponseReceived(string owner, bool response)
{
    if (response)
    {
        CreateAndSetupStreamReceiver(owner, MediaType.Audio);

        try
        {
            streamSender.AttachMicrophone(microphone);
        }
        catch (Exception ex)
        {
            MessageBox.Show(ex.Message);
        }

        audioEnableWith = owner;
        txtChatLog.Text += String.Format("{0} accepted your audio request.\n", owner);
        btnAudio.Content = "Disable audio";
        audioIsEnable = true;
    }
    else
    {
        txtChatLog.Text += String.Format("{0} rejected your audio request.", owner);
    }
}

Code 19 - The audio request is answered with an audio response from the other end

If the request is accepted, the requested media (in this case the microphone) is matched to the existing MediaStreamSender (streamSender) object. This way, the voice arriving from the microphone is forwarded to the server. (It needs to be mentioned that this part does not bother with extracting and compressing audio data from the microphone because Ozeki VoIP SIP SDK does these processes for us via its MediaStreamSender and Microphone objects.)

Besides setting the input media device, it is also necessary to set the device that is responsible for playing, in other words, to set the playing of the mediaStream that is published with the other party's ID (Code 20).

public void CreateAndSetupStreamReceiver(string playerName, MediaType mediaType)
{
    if (streamReceiver == null)
    {
        streamReceiver = new MediaStreamReceiver(connection);
        streamReceiver.StreamStateChanged +=
            new EventHandler<GenericEventArgs<StreamState>>(streamReceiver_StreamStateChanged);
        streamReceiver.Play(playerName);
    }

    try
    {
        if (mediaType == MediaType.Audio)
            streamReceiver.AttachAudioPlayer(audioPlayer);
        else
            streamReceiver.AttachVideoPlayer(videoPlayerControl);
    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.Message);
    }

}

Code 20 - The initialization of the stream receiver object

Here an appropriate player (audioPlayer/videoPlayerControl) is matched based on the media type. Again, this part does not deal with decoding the media because Ozeki VoIP SIP SDK does this process for us.

Clients process incoming media requests in AudioRequestReceived and CameraRequestReceived method (Code 21). Since similarity is high between the two, here only audio is demonstrated.

public void AudioRequestReceived(string owner, bool isEnable)
        {
            if (isEnable)
            {
                winMediaReq = new CWindowCameraRequest(owner, MediaType.Audio);
                winMediaReq.ResponseSelected += new EventHandler(winMedia_ResponseSelected);
                winMediaReq.Show();
            }
            else//close audio conversation
            {
                streamSender.DettachMicrophone();
                DestroyStreamReceiver();
                txtChatLog.Text += String.Format("{0} disabled his/her microphone.\n", owner);
                btnAudio.Content = "Enable audio";
                audioIsEnable = false;
            }
        }

Code 21 - The AudioRequestReceived client-side method

The request for media chat and the termination of it is also indicated in this method. The two functions are differentiated with the isEnable parameter. If this parameter is true, it is a media initiation. In this case, the given media request will be accepted or rejected with the help of the CWindowCameraRequest window. The response that is selected in this window will be processed by the winMedia_ResponseSelected event handler method (Code 22). If the isEnable parameter indicates the termination of the communication with the given media (false value), the input and player devices that participate in the communication can be detached if it is needed.

The winMedia_ResponseSelected sends back the response of the incoming media request to the other client. In case of positive response (it is accepted) it does the setup of the communication with the given media:

void winMedia_ResponseSelected(object sender, RequestResponseEventArgs e)
{
    winMediaReq.ResponseSelected -= new EventHandler(winMedia_ResponseSelected);
    if (e.Type==MediaType.Video)
        connection.InvokeOnConnection("SendCameraResponse", lblNickName.Text, e.Owner, e.Response);
    else
        connection.InvokeOnConnection("SendAudioResponse", lblNickName.Text, e.Owner, e.Response);

    if (e.Response)
    {
        switch (e.Type)
        {
            case MediaType.Video:
                 txtChatLog.Text += "Camera request accepted\n";
                 CreateAndSetupStreamReceiver(e.Owner, MediaType.Video);

                try
                {
                    streamSender.AttachCamera(cameraRecorderControl);
                }
                catch (Exception ex)
                {
                    MessageBox.Show(ex.Message);
                }

                btnCamera.Content = "Disable video chat";
                cameraIsEnable = true;
                cameraEnableWith = e.Owner;
                break;
            case MediaType.Audio:
                 txtChatLog.Text += "Audio request accepted\n";

                try
                {
                    streamSender.AttachMicrophone(microphone);
                }
                catch (Exception ex)
                {
                    MessageBox.Show(ex.Message);
                }

                 CreateAndSetupStreamReceiver(e.Owner, MediaType.Audio);
                btnAudio.Content = "Disable audio chat";
                audioIsEnable = true;
                audioEnableWith = e.Owner;
                break;
        }
    }
    else
    {
        txtChatLog.Text +=String.Format("{0} request rejected\n",e.Type);
    }
}

Code 22 - The EventHandler method for media response selection

In case of positive response it matches the input device related to the given media with the MediaStreamSender object and couples the player devices (that are required for playing the incoming media) with MediaStreamReceiver.

After the proper matching, the two clients start to play to each other the recorded media data. If one of the clients breaks the connection, the other party is notified via the server.

Summary

The purpose of this example is to demonstrate how media data can be transferred with the use of Ozeki VoIP SIP SDK. For this reason, there may be cases that are not handled properly, however, these cases does not affect the fact that with the use of Ozeki VoIP SIP SDK, developers can develop the requested application faster, simpler and more efficiently.

For more information, please contact us at info@voip-sip-sdk.com.