0 Preface
The DirectsHow application framework completes the underlying work of streaming media processing, so that programmers do not need to care about how to input data and how to output after processing, but only need to care about how to process the input data. The H.264 video codec standard has a high compression ratio and excellent network affinity, and is generally regarded as the most influential streaming media video compression standard. The streaming media player that combines Direct-show and H.264 will undoubtedly have very good performance.
1Directshow technology and H.264 video compression standard introduction
Directshow is a set of streaming media development software packages provided by Microsoft Corporation. It provides a complete solution for multimedia applications that handle high-performance multimedia file playback and audio and video capture on the Windows platform.
Directshow is a complete COM-based application system, which is located in the application layer. It uses the FilterGraph model to manage the entire data flow processing process; each function module involved in data processing is called Filter; each Filter in FilterGraph according to a certain Sequentially connected into a "pipeline" to work together. Filter is a COM component, and its functions can be implemented by users. DirectShowSDK also provides users with some standard Filters for users to use. Filters are connected to each other in the FilterGraph through Pin in a certain order. Pin is also a COM object.
H.264 is the latest international video coding standard jointly developed by the joint development group of ITU-T and ISO / IEC. The H.264 / AVC video coding standard has significantly improved coding quality and compression ratio compared with the original video coding standard. On the same visual perception quality, the coding efficiency is about 50% higher than H.263, MPEG-2 and MPEG-4. H.264 not only has excellent compression performance, but also has good network affinity. Therefore, H.264 is generally regarded as the most influential streaming media video compression standard.
2 System Design Framework
This system is based on the Direotshow application framework and the H.264 video compression standard, and realizes the function of receiving streaming media data from the network and playing it on the client in real time. The streaming media file is an AVI file encoded with H.264. Since directshow provides AVISplitteRFilter, AudioDecoder and standard Video / AudioRenderer, this system only needs to design custom network source Filter and H.264 decoding Filter.
Multimedia streaming actually involves two technologies. One is the communication technology between the server and the client, including the transmission of multimedia data, command control, etc .; the second is the technology that the client decodes and plays the received multimedia stream in real time. Obviously, Windows socket technology can be used for network communication, and direct-show technology can be used for decoding and playing of multimedia streams. This article uses the direetshow application framework, designed the network source filter and h.264 decoding filter, and built a streaming media player through FilterGraph.
FilterGraph playing local AVI files based on H.264 encoding is shown in Figure 1. Simply replace the local FileSourceFilter with the network source Filter and the CoreAVCVideoDe-coder with the h.264 decoding Filter to implement the network H.264 video player.
3 System design and implementation
3.1Filter design process
The coding implementation of Filter includes the registration information of Filter, the realization of frame functions on Filter, the realization of logic control class, the realization of custom interface, the realization of property page, the protection of property rights, etc.
First, we must analyze the function of Filter and its position in FilterGraph to determine the filter model and select an appropriate base class; then, define the input and output PIN and custom interface, register the Filter information; and finally implement the base All pure virtual functions and custom interface functions of the class, and rewrite the related functions of the base class to customize the filter function.
3.2 Design of Network Source Filter
The main function of the source Filter is to receive the streaming media data from the server and provide it to other Filter in the FilterGraph.
Since the AVIsplitter included in directshow works in pull mode, the source Filter also works in pull mode.
This source Filter uses double-buffered circular queue technology to achieve data reception and transfer data to the next level Filter. There are several reasons for using this technology:
(1) During the connection between SourceFilter and SplitRer, a part of data will be read from SourceFilter to obtain the format description of the data, otherwise FilterGraph cannot complete the connection. Before the Source Filter and SplitterFilter are connected, a waiting thread should be started. After the SourceFilter data cache has received enough data in advance, a complete FilterGraph will be built.
(2) When the complete FilterGrapH is built and is in a running state, SourceFilter must dynamically receive data and continuously provide new data to SplitteRFilter. The double-buffered circular queue not only makes full use of memory space, but also provides a stable data source.
(3) The buffer queue can stabilize the code rate and effectively reduce the impact of network delay, blocking and jitter.
The working process of the source Filter is as follows: establish a circular buffer queue, the queue tail pointer is used to buffer the data received from the network, the queue head pointer is used by Splitter to read the data, separate the audio and video, and pass it to the next level of Decoder processing; After receiving the network data, insert the data into the tail of the queue and move the tail pointer backward; when Splitter needs to read the data, read the data from the head of the queue and move the head pointer backward.
The streaming media transmission adopts the client / server architecture. There is also a Socket communication protocol problem between the server and the client. Since the streaming media is continuous, its synchronization point cannot be selected at will. Therefore, in order to transmit streaming media data, it must be used. Connection-oriented reliable transport protocol (TCP). Client and server control and feedback messages can be transmitted using (UDP). The server first creates a listening socket to listen for connection requests from the client. Once listening to the client's request, the server creates a Socket for data transmission and binds it to the client requesting the connection. At this time, the server is in the ready state for data transmission. When the client issues a command, the server performs the corresponding operation according to the type of command, such as data sending, stopping, and disconnection.
On the server side, the continuous H.264 stream is first divided into small pieces of load data, and an information header is added together to send using the TCP protocol; on the client side, the load data of the small packets are assembled according to the description of the information header, and then H .264 decoding and playback. Socket data transmission structure is: load type (8bit), load data length (16bit) and load data packet (2324Byte).
The client implementation can refer to the MEMFileFilter in the SDK, directly copy the four files asyncrdr.cpp, asyncrdr.h, asyncio.cpp, asyneio.h from the SDK example, and then derive the stream processing class CMemStream from CAsyncStream in the MemFilter.h file , Customize the data source and read the data; derive the filter class CMemReader from CAsyncReader, implement the SourceFilter framework with 1 output PIN, and complete the connection with SplitterFilter.
The data flow between network source filter classes is shown in Figure 2, and the data flow of the next level Filter "pull" thread is shown in Figure 3. Among them: CAsyneStream is used to mark the data stream; CAsyneRequest marks the input and output requests; CAsyncIo realizes the control of data input and output. Programming mainly implements CMemStream: Read () function to read data from a specific data source to the Sample of the next level Filter.
Lithium Polymer Charger,36V Lithium Battery Charger,36V Bike Lithium Battery Charger,36V Bicycle Lithium Battery Charger
Changxing Deli Technology Co., Ltd. , https://www.delipowers.com