Search Results

Naardic is online fitness training platform Cheers people, this is Vaibhav from Agilis AS, I am currently heading the team of developers focused on making a custom video conferencing system for our product Naardic, which is an realtime interactive fitness platform, this blog is about a few challenges faced while building our custom video conference. What we needed ? Before we started building our own platform we were dependent on 3rd party tools for our interactive classes like zoom. So the benchmark was clear to us, it had to be stable as third party platform like zoom. Most importantly trainers were used to seeing 30+ people in the single screen, which was another major challenge. Another challenge, testing ? Once we knew what we wanna build it was time to build the idea & test it. It was clear from the start, it was a challenge we had to coordinate testing video call by inviting no. of participants testing, that being said it wasn't feasible at all to invite 60+ no. of participants to test we had to resort to some automation for testing. More Testing .... It's easier said than done, what we desired to & what we do never aligns There were multiple CPASS vendors we tested in order to find the right fit of us with the right bandwidth & the right amount of people to accommodate on a single screen, although this looks like the only thing we needed to access, but the testing included a lot of other quality metrics and how to evaluate it, most of them would be covered in the next blog . The next blog would define what & how you can measure call quality.

Rails is a popular web application framework for building robust and scalable web applications. Daily.co is a platform for building real-time video and audio applications, including video conferencing. In this blog, we will walk through the steps to create a video conferencing system using Rails and Daily.co. Step 1: Create a new Rails application The first step is to create a new Rails application using the command line tool. Open the terminal and run the following command: rails new videoconferencing --database=postgresql This command creates a new Rails application named "videoconferencing" with PostgreSQL as the default database. Step 2: Add HTTP Party SDK Next, add the HTTP Party Gem to the application to make api calls to daily to create room. Open the `Gemfile` in the root directory of the application and add the following line: gem 'httparty' Save the file and run `bundle install` in the terminal to install the gem. Step 3: Configure Daily.co API credentials To use the credentials file to store api_key. Use this command to add `EDITOR=nano rails credentials:edit` : daily: access_token: YOUR_API_KEY Replace `YOUR_API_KEY` with your Daily.co API Key which you can get from daily developers section `https://dashboard.daily.co/developers` . Step 4: Create daily api service for creating video conference room To create a new video conference room, create a folder named services in app folder, inside `services` create another folder `dailyco`, inside `dailyco` create a file named `room.rb` and add the following code in room.rb : class Dailyco::Room include HTTParty base_uri 'https://api.daily.co/v1' def initialize(name) @name = name end def create options = auth_headers body = {name: @name} self.class.post('/rooms/', options.merge({body: body.to_json})) end def auth_headers access_token = Rails.application.credentials.config[:daily][:access_token] { headers: { 'Content-Type' => 'application/json', 'Authorization' => "Bearer #{access_token}", } } end end Step 5: Create a new video conference room To create a new video conference room, create a new controller named `video_conferences_controller.rb` in the `app/controllers` directory or you can use command `rails g controller VideoConferences` and add the following code : class VideoConferencesController < ApplicationController def create daily = Dailyco::Room.new(params[:name]) room = JSON.parse daily.create redirect_to room['url'] end end This code creates a new video conference room with the name provided in the `params` hash and redirects the user to the room URL. Step 6: Create a view to display the video conference form Create a new view named `new.html.erb` in the `app/views/video_conferences` directory and add the following code: Create a new video conference room This code displays a form to create a new video conference room and submit it to the `create` action of the `VideoConferencesController`. Step 6: Start the Rails server Start the Rails server using the following command, if you face database not found error you can create db using command `rake db:create`: rails server Open the browser and navigate to `http://localhost:3000/video_conferences/new`. This should display the video conference form. Enter a name for the video conference room and click on the "Create Room" button. Step 7: Join the video conference To join the video conference, click on the link provided on the next page. This should open the video conference in a new tab. You can share this link with other participants to join the video conference. In conclusion, Daily.co provides a powerful API for building real-time video and audio applications, including video conferencing. By integrating into a Rails application, you can easily create a video conferencing system that is scalable and customizable. With the steps outlined in this blog, you can create a simple video conferencing system using Rails and Daily.co in no time.

WebRTC is an open-source project that enables real-time communication between web browsers and mobile applications via simple APIs. Testing WebRTC video calls is essential to ensure a good user experience. Chrome WebRTC dumps provide a useful tool for analyzing and testing the quality of WebRTC video calls. Here are the steps to test a WebRTC video call using Chrome WebRTC dumps: 1. Install the Chrome WebRTC Internals extension(It's an old extension last updated on 21 Dec 2021 ): The first step is to install the Chrome WebRTC Internals extension from the Chrome Web Store. This extension provides access to the Chrome WebRTC debugging and diagnostic tools. 2. Enable the Chrome WebRTC diagnostic logging: To enable the Chrome WebRTC diagnostic logging, type chrome://webrtc-internals in the Chrome address bar and press Enter. Click on the "Start" button next to "Event Log." This will enable the diagnostic logging for all WebRTC events in Chrome. 3. Record the WebRTC dump: Once you have enabled the diagnostic logging, start the WebRTC video call and let it run for a few minutes. After the call, stop the diagnostic logging by clicking on the "Stop" button next to "Event Log." This will generate a WebRTC dump that contains all the data packets transmitted during the call. 4. Analyze the dump: Once you have the dump, you can analyze it using the Chrome WebRTC diagnostic tools. Click on the "PeerConnection" tab to see detailed information about the WebRTC connection. Look for packet loss, jitter, and latency, which are the key metrics that impact call quality. 5. Calculate metrics: Calculate the key metrics, including Mean Opinion Score (MOS), packet loss rate, jitter, and delay. MOS is a subjective quality metric that rates call quality on a scale of 1 to 5, with 5 being the best. Packet loss rate measures the percentage of data packets lost during the call. Jitter measures the variability in the delay of data packets. Delay measures the time it takes for data packets to travel from the sender to the receiver. 6. Interpret the results: Interpret the results and identify any issues that impact call quality. For example, high packet loss, jitter, or latency can result in poor call quality, including choppy video or delays. Use the results to optimize the network, troubleshoot issues, and improve call quality. 7. Repeat the test: Repeat the test to ensure that the changes you made have improved call quality. Test the call under different network conditions, such as high traffic or low bandwidth, to ensure that it works well in all scenarios. Chrome WebRTC dumps provide a useful tool for testing WebRTC video calls. By enabling diagnostic logging and analyzing the dump, you can identify issues that impact call quality and optimize the network to improve the user experience. Repeat the test under different network conditions to ensure that the call works well in all scenarios.

WebRTC (Web Real-Time Communication) is an open-source project that enables real-time communication between web browsers and mobile applications via simple APIs. It allows developers to build voice, video, and data communication applications directly into web browsers and mobile devices without the need for plugins or native applications. One of the most popular use cases for WebRTC is building media servers, which enable communication between multiple clients. Media servers are a crucial component of WebRTC-based applications that enable developers to build rich real-time communication experiences. There are several media servers available with WebRTC tech, each with its own set of features and capabilities. Here are some of the popular media servers available with WebRTC tech: 1. Kurento Media Server: Kurento Media Server is an open-source media server that provides support for WebRTC, RTP, and HTTP-based protocols. It enables developers to build complex real-time communication applications, such as video broadcasting, video conferencing, and augmented reality. Kurento offers a powerful set of APIs that enable developers to create custom media processing pipelines and add advanced features like computer vision, speech recognition, and augmented reality. 2. Jitsi Videobridge: Jitsi Videobridge is a WebRTC-compatible media server that provides support for group video conferencing, online classrooms, and webinars. It supports up to 75 participants per session and offers advanced features like scalable video coding, adaptive bitrate, and simulcast. Jitsi Videobridge is an open-source project that is available under the Apache License 2.0. 3. Janus Gateway: Janus Gateway is a general-purpose WebRTC server that supports audio, video, and data channels. It provides a simple and flexible API that enables developers to build custom real-time communication applications. Janus Gateway offers support for multiple audio and video codecs, including H.264, VP8, and Opus. It is available under the GPL license. 4. Wowza Streaming Engine: Wowza Streaming Engine is a media server that provides support for WebRTC, HLS, RTMP, and other streaming protocols. It offers a powerful set of features that enable developers to build live streaming applications for web, mobile, and TV platforms. Wowza Streaming Engine provides support for transcoding, recording, and playback of live streams. 5. Red5 Pro: Red5 Pro is a WebRTC-compatible media server that provides support for live streaming, real-time messaging, and recording. It offers low-latency communication for multiplayer gaming, e-learning, and interactive broadcasting. Red5 Pro provides a robust set of APIs that enable developers to build custom real-time communication applications. Media servers are an essential component of WebRTC-based applications that enable developers to build rich real-time communication experiences. There are several media servers available with WebRTC tech, each with its own set of features and capabilities. Choosing the right media server depends on the specific needs of the application and the development team's skillset.

There are five different types of augmented reality which broadly lie in two broad categories. The two broad categories are Trigger-Based AR and View-Based AR. Trigger-based AR has a characteristic that it gets triggered or initiates whenever it found the trigger. A Trigger can be anything like a 2D picture printed on a paper or a 3D model or a specific location (GPS location) etc. The second form of augmented reality is View-Based AR which renders the 3D content without the reference to what is in the view. Let’s try to understand the different types of augmented reality in more detail. Marker Based AR — Paper Comparison of Marker-based (paper) AR using AR Kit and Vuforia 2D marker-based AR or marker-based AR-paper is also known as Image recognition based AR. Here Markers are usually distinct visual patterns that are distinct from the environment and can be easily recognized and processed by the camera. Usually, in marker-based augmented reality, AR is triggered whenever a marker is recognized based on some pre-defined logic. In the case of 2D marker-based AR, these markers are images printed on paper. A very well-known example of this AR is done by Esquire Magazine asking the user to scan the front page which brings Robert Downet Jr Comes live on-page. Marker Based AR — Object Marker-Based AR (Object) using AR Kit This form of AR is commonly known as Object Detection-based AR. As stated above marker-based AR requires a marker to trigger the AR. In the case of object-based marker-based AR, augmented reality is triggered with the help of a real 3D object. For this, the 3D structure of the object acts as a marker. This type of AR gives rise to another form of AR commonly known as Superimposition Based AR. As the name suggests it is the superimposition of the object. AR provides a new and replaced view of the real-world object which acted as a marker. Location Based AR Wikitude Drive AR navigation system, Augmented reality mobile navigation Location-based AR ties AR to a specific place. Virtual 3D objects or information are anchored to location and then displayed according to user input when a user enters to that location. Pokémon Go is a very popular example of location-based Augmented Reality. Another good example of location-based AR is AR navigation and tourist guide. Marker Less AR Comparison of Marker-less AR using AR Kit and Vuforia. This type of AR is more versatile than marker-based AR, it doesn’t require any marker 2D or 3D object to trigger. It relies on the positional information gathered from the device using a camera, GPS, digital compass, accelerometer. These data inputs build a 3D understanding of the realtime environment around the user using a process known as SLAM (Simultaneous Localisation and Mapping) and depth prediction. Projection Based AR Live demo of projection-based AR. Advanced projectors can be used to create an immersive new layer of information over a flat surface. It doesn’t require user-operated technology. It provides a new futuristic experience. The widespread uses of projection-based AR techniques can be used to create deception about the position, orientation, and depth of an object.

In today’s world AR can be displayed on various devices including screens, glasses, mobile phone (iOS, Android etc.), hand-less devices, head-mounted devices etc. AR uses different technologies like depth tracking, SLAM (Simultaneous Localisation And Mapping) etc.¹ AR uses computer vision to understand the world around the user using the camera feed on which digital content including computer graphics are rendered over the real-time objects. Detecting the features of the real world by applying the hit test and other SLAM techniques. Based on the result of the SLAM technique recognising the real world around the user. Let’s take an example to understand how exactly AR works. Let's take an example for mobile-based AR game to scan a box. Firstly the mobile phone camera is used to capture the real world around the user. For a human, it’s a very easy task to understand the real world around them or to recognise the box, but for the computer, it’s not the same, there is a whole field of Computer Science dedicated for it, commonly known as Computer vision. The raw data from the camera is processed using computer vision to recognize the box.Once the box is recognized the game is triggered which in turn will render the augmented 3D object, making sure that it preciously overlap with the box. AR requires the understanding of the real-world in term of “Semantics” and “Geometry”. Semantics means what is that object and geometry mean where is that object. The semantics of that image can be understood by the help of deep learning which recognises the object irrespective of the 3D geometry. Computer Vision helps in recognizing the 3D from a 2D image. For precious overlapping, the 3D orientation of the box and position of the box is captured using computer vision. Mostly Augmented Reality is live, which means all the process stated above happens every time for each new frame. For most of the mobile phone, the camera works at 30 frames per second, which means all the complete process stated above needs to complete within 30 milliseconds each time. Sometimes because of intensive processing, there can be a delay of 50 milliseconds but over brain does not notice that. Making this complete process fast and reliable. Components of Augmented Reality Source: https://pixabay.com/photos/camera-inside-mechanics-technician-943244/ Let’s try to understand the different component which helps in achieving it. Camera and Sensors: For collection, the data of user interaction and the real-world around the user, camera live feed and data from different sensors are used including gyro meter, accelerometer, etc. The system can also use the special camera as in the case of HoloLens’s or normal camera of the smartphone to get camera feeds. Processing: Once real-world data has been collected it needs to be processed in real-time. For this most of the type of AR, the devices act like a small computer and required great processing capabilities. Similar to the computer they also require a CPU, GPU, RAM, GPS, Wi-Fi, flash memory, etc. Projection: Once the data is processed, 3D content needs to rendered and projected in the real-time on to a surface to view. This is one field in which researches are going on across the globe. Reflection: Proper image alignment is performed using reflection. For this most of the devices uses mirrors that help human eyes to view and understand the 3D content which is rendered in correct alignment. Some of the devices use a double mirror while other devices use an array of curved mirrors which reflect light to a camera and in turn to the human eyes. [1]: R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier and B. MacIntyre, “Recent advances in augmented reality,” in IEEE Computer Graphics and Applications, vol. 21, no. 6, pp. 34–47, Nov.-Dec. 2001. DOI:10.1109/38.963459

Thomas Caudell and David Mizell widely considered as the people who coined the term “Augmented Reality” in 1992¹. They both were the researcher in Boeing and they used this term for the first time when Augmented Reality was getting used in Boeing for assembling of aircraft parts. The conception of augmented reality first occurred during the start of the 19th century. In 1901 Frank L Baum used the concept of augmented reality in one of his novel “The Master Key” which states the use of an electronic glasses which were used by people to maps data, this device was called “character marker”². To some extent augmented reality was achieved in 1957 by a cinematographer and filmmaker Morton Heilig as shown in Figure below. Source: Sensorama which uses visual, sound, vibrations and smells to enhance User Experience by Morton Heilig in 1957¹. He developed a Sensorama which uses visual, sound, vibrations and smells to enhance user experience. In 1962 he filed a patent for Sensorama Simulator “apparatus to stimulate the senses of an individual to simulate an experience realistically”. He developed this purely for the purpose of entertainment and gives the user the experience of riding a motorbike through the city of Brooklyn. Source: “The Sword of Damocles” was the first head-mounted AR device developed by Ivan Sutherland and his team in 1968³ In 1968, Harvard Professor Ivan Sutherland along with his student Bob Sproul developed the first augmented reality device which he named as “The Sword of Damocles”³ as shown in Figure above. The sword of Damocles was a head-mounted device that was hung from the ceiling. The user was able to see computer-generated graphics which were simple wireframe drawing. This makes the user experience an alternative virtual reality. In 1974 Myron Krueger built an “artificial reality” laboratory called the Video place, it combined projector with video camera hat emitted onscreen silhouettes, surrounding users in a very interactive environment. Video place allows the user to interact with virtual objects in real-time. In 1980 “wearable computing” term was coined by Steve Mann who was a computational photography researcher. In 1987 Douglas George and Robert Morris developed the prototype of a heads-up display (HUD) which was used to display astronomical data over the real sky. In 1989 “Virtual Reality” term was coined by Jaron Lanier and finally, in 1990 Tom Caudell coined the term “Augmented Reality”. In 1992 Louis Rosenberg developed one of the earliest functioning AR systems for Air Force widely known as Virtual Fixture. It allowed the military to control the virtually guided machinery to perform the task in remote areas. In 1994 the first AR theatre production was created by Julie Martin⁴. It featured acrobats who danced within and around virtual objects on their real stage. This was the time AR started making its way in the entertainment industry as technology advancement. Recent Research in Augmented Reality What would the world look like if we could derive our identity from a more interactive form of communication that merges the digital and the physical? In 1998 the 1st and ten line computer systems started broadcasting by Sports vision, casting the first down marker during a live NFL game. In 1999 naval researchers started their research on Battlefield Augmented Reality System or BARS. These were the early model of wearable units for the soldiers. In the same year, NASA-X 38 spacecraft was launched using the augmented reality system. They used a Hybrid Synthetic Vision System which overlay the map data to enhanced visual navigation. The year 2000 was a benchmark in the history of Augmented Reality, in this year Hirokazu Kato developed an open-source software library commonly known as AR Toolkit which uses video tracking to overlay computer graphics on a video camera. AR Toolkit is a widely used library for the development of augmented reality and mixed reality product even today. In 2004 Trimble Navigation presented an outdoor helmet-mounted AR system. Source: https://www.wikitude.com/showcase/wikitude-navigation/ In 2008 Wikitude made the first augmented reality-based travel guide for android mobiles. In the year 2009 AR Toolkit brought the augmented reality to the web browsers. In the same year augmented reality entered the printed media sector. Esquire Magazine promoted AR, asking the user to scan the front page which brings Robert Downet Jr Comes live on-the-page. After this AR hardware and software makes the leap towards consumer audiences. In 2013 car manufactures started using augmented reality as the new sage vehicle service manual. A very good example of this was Volkswagen MARTA app which was developed to provide service technicians with a step by step technical assistant to foresee how to perform a specific task on a vehicle in front of them. In the same year, google successfully tested google glass which is multi-purpose augmented reality support. In 2014 Magic Leap announces a large investment of $50M in augmented reality. ARKit vs ARCore vs Vuforia In the same year, Google announces shipment of Google Glasses to the customers which actually started the trend wearable AR. In 2015 Apple launched AR Kit which is a software development toolkit for augmented reality for iOS platform. In the same year, Vuforia launched a cross-platform augmented reality software development toolkit which supports 2D, 3D marker-based AR as well as marker-less AR. By 2016 Augmented Reality and Virtual Reality total investment reached above $1.1 billion. In the same year, Microsoft launched the Microsoft HoloLens’s which brought a revolution in this industry bringing mixed reality in every one hand, which in turn open thousands door for innovation and development in this industry. Source:https://pixabay.com/photos/pokemon-go-pokemon-street-lawn-1569794/ In 2016 Niantic published Pokémon Go which is an augmented reality-based mobile game on iOS and Android. This game completely changed the way how people think about augmented reality and became a huge success with more than 1 billion downloads worldwide. This game blew the world and earned more than $2 million within 1 week of launch. In 2018 Google launched AR Core which is a software development toolkit that allows augmented reality application development for android. [1]: Krevelen, D. W. F. van and Ronald Poelman. “A Survey of Augmented Reality Technologies, Applications and Limitations.” (2010). [2]: The Master Key, L. Frank Baum, Fanny Y. Cory, Bowen-Merrill, 1901 [3]: ’The Sword Of Damocles’, 1st Head Mounted Display, September 11, 2016,3 Rock AR https://vrroom.buzz/vr-news/guide-vr/sword-damocles-1st-head-mounted-display [4]: Brad A. Myers. 1998. A brief history of human-computer interaction technology. interactions 5, 2(March 1998), 44–54. DOI=http://dx.doi.org/10.1145/274430.274436

Augmented Reality is not a new concept, it has come a long way from science fiction films to a science-based reality. The oxford dictionary defines Augmented Reality as follows A technology that superimposes a computer-generated image on a user’s view of the real world, thus providing a composite view. The word Augmented comes from augmenting which means to add or enhance something. In the case of augmented reality, sound, 3D graphics, touch feedback etc. are added to the real world to enhance user experience¹. “Augmented Reality is a variation of the virtual environment or more commonly known as virtual reality.”² In Virtual Environment, a user is completely immersed into a virtual world and unable to see the real-world whereas augmented reality allows the user to see the real world and virtual objects suspended over it. It allows the virtual objects and the real objects co-exist in the same view. Milgram and Kishino in 1994 were the first persons who clearly formulated this taxonomy and presented the clean terminology of co-existence of virtual and real-world within Reality-Virtuality Continuum³. The terminology of co-existence of the virtual and real-world within Reality-Virtuality Continuum by Milgram and Kishino³ Augmented Reality Vs Virtual Reality Vs Mixed Reality Source: https://www.flickr.com/photos/lenovophotolibrary/40321592225/ Augmented Reality(AR) and Virtual Reality(VR) are two ways in which technology can change the way we look at the world today. Although they are very much similar to each other they are not at all the same term. Augmented Reality is defined as follows: “An enhanced version of reality created by the use of technology to add digital information to something”⁴ Whereas virtual reality is defined as “The use of technology to create a simulated world”⁴ The main difference between augmented reality and virtual reality is that in VR a user is completely inside the virtual world, they do not know what goes in the real world whereas in AR the virtual objects are layered over the real-time objects. A very good example of AR is Pokémon Go or Snapchat lenses using your smartphone. A user can immerse him/her in virtual reality using VR devices such as HTC vibe, Oculus Rift etc. On combining experience of augmented reality with virtual reality, mixed reality experience is created. In mixed reality, the user can interact with the real world as well as virtual objects. It is a new concept and Microsoft HoloLens is a very good example of it⁵. Introducing Microsoft HoloLens 2 [1]: B. Bell, S. Feiner and T. Hollerer, “Information at a glance [augmented reality user interfaces],” in IEEE Computer Graphics and Applications, vol. 22, no. 4, pp. 6–9, July-Aug. 2002. DOI:10.1109/MCG.2002.1016691 [2]: A Survey of Augmented Reality by Ronald T. Azuma, Hughes Research Laboratories, In Presence: Teleoperators and Virtual Environments 6, 4 (August 1997), 355–385 [3]: Paul Milgram. A taxonomy of mixed reality visual displays. 1994. [4]: M. Braitmaier and D. Kyriazis, “Virtual and Augmented Reality: Improved User Experience through a Service Oriented Infrastructure,” 2011 Third International Conference on Games and Virtual Worlds for Serious Applications, Athens, 2011, pp. 40–46. DOI: 10.1109/VS-GAMES.2011.12 [5]: WHAT’S THE DIFFERENCE BETWEEN AR, VR, AND MR?, The Franklin Institute, 2018, https://www.fi.edu/difference-between-ar-vr-and-mr