ios – 从CMBlockBuffer中提取h264

函数CMBlockBufferGetDataPointer可以访问所需的所有数据,但是有一些不太明显的事情需要将其转换为基本流.

AVCC与附件B格式

CMBlockBuffer中的数据以AVCC格式存储,而基本流通常遵循附件B规范(here是两种格式的优秀概述).在AVCC格式中,4个第一个字节包含NAL单元的长度(H264数据包的另一个字).您需要使用4字节起始码替换此标头：0x00 0x00 0x00 0x01,用作附件B基本流中的NAL单元之间的分隔符(3字节版本0x00 0x00 0x01也可以正常工作).

单个CMBlockBuffer中有多个NAL单元

下一个不是很明显的事情是,单个CMBlockBuffer有时会包含多个NAL单元.苹果似乎在每个I帧NAL单元(也称为IDR)中添加了一个包含元数据的附加NAL单元(SEI).这可能是为什么您在单个CMBlockBuffer对象中看到多个缓冲区.但是,CMBlockBufferGetDataPointer函数为您提供了访问所有数据的单个指针.话虽如此,多个NAL单元的存在使AVCC头的转换变得复杂.现在,您实际上必须读取AVCC标题中包含的长度值以找到下一个NAL单元,并继续转换标题,直到达到缓冲区的结尾.

Big-Endian与Little-Endian

下一个不是很明显的事情是,AVCC头是以Big-Endian格式存储的,而iOS是本机的Little-Endian.因此,当您读取AVCC头文件中包含的长度值时,首先将其传递给CFSwapInt32BigToHost函数.

SPS和PPS NAL单位

最后不太明显的是,CMBlockBuffer内的数据不包含参数NAL单元SPS和PPS,其中包含解码器的配置参数,如配置文件,级别,分辨率,帧速率.这些作为元数据存储在样本缓冲区的格式描述中,可以通过CMVideoFormatDescriptionGetH264ParameterSetAtIndex函数进行访问.请注意,您必须在发送之前将起始码添加到这些NAL单元. SPS和PPS NAL设备不必每个新的帧发送.解码器只需要读取一次,但通常会周期性重新发送它们,例如在每个新的I帧NAL单元之前.

代码示例

以下是考虑所有这些事项的代码示例.

static void videoFrameFinishedEncoding(void *outputCallbackRefCon,void *sourceFrameRefCon,Osstatus status,VTEncodeInfoFlags infoFlags,CMSampleBufferRef sampleBuffer) {
    // Check if there were any errors encoding
    if (status != noErr) {
        NSLog(@"Error encoding video,err=%lld",(int64_t)status);
        return;
    }

    // In this example we will use a NSMutableData object to store the
    // elementary stream.
    NSMutableData *elementaryStream = [NSMutableData data];


    // Find out if the sample buffer contains an I-Frame.
    // If so we will write the SPS and PPS NAL units to the elementary stream.
    BOOL isIFrame = NO;
    CFArrayRef attachmentsArray = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer,0);
    if (CFArrayGetCount(attachmentsArray)) {
        CFBooleanRef notSync;
        CFDictionaryRef dict = CFArrayGetValueAtIndex(attachmentsArray,0);
        BOOL keyExists = CFDictionaryGetValueIfPresent(dict,kCMSampleAttachmentKey_NotSync,(const void **)&notSync);
        // An I-Frame is a sync frame
        isIFrame = !keyExists || !CFBooleanGetValue(notSync);
    }

    // This is the start code that we will write to
    // the elementary stream before every NAL unit
    static const size_t startCodeLength = 4;
    static const uint8_t startCode[] = {0x00,0x00,0x01};

    // Write the SPS and PPS NAL units to the elementary stream before every I-Frame
    if (isIFrame) {
        CMFormatDescriptionRef description = CMSampleBufferGetFormatDescription(sampleBuffer);

        // Find out how many parameter sets there are
        size_t numberOfParameterSets;
        CMVideoFormatDescriptionGetH264ParameterSetAtIndex(description,NULL,&numberOfParameterSets,NULL);

        // Write each parameter set to the elementary stream
        for (int i = 0; i < numberOfParameterSets; i++) {
            const uint8_t *parameterSetPointer;
            size_t parameterSetLength;
            CMVideoFormatDescriptionGetH264ParameterSetAtIndex(description,i,&parameterSetPointer,&parameterSetLength,NULL);

            // Write the parameter set to the elementary stream
            [elementaryStream appendBytes:startCode length:startCodeLength];
            [elementaryStream appendBytes:parameterSetPointer length:parameterSetLength];
        }
    }

    // Get a pointer to the raw AVCC NAL unit data in the sample buffer
    size_t blockBufferLength;
    uint8_t *bufferDataPointer = NULL;
    CMBlockBufferGetDataPointer(CMSampleBufferGetDataBuffer(sampleBuffer),&blockBufferLength,(char **)&bufferDataPointer);

    // Loop through all the NAL units in the block buffer
    // and write them to the elementary stream with
    // start codes instead of AVCC length headers
    size_t bufferOffset = 0;
    static const int AVCCHeaderLength = 4;
    while (bufferOffset < blockBufferLength - AVCCHeaderLength) {
        // Read the NAL unit length
        uint32_t NALUnitLength = 0;
        memcpy(&NALUnitLength,bufferDataPointer + bufferOffset,AVCCHeaderLength);
        // Convert the length value from Big-endian to Little-endian
        NALUnitLength = CFSwapInt32BigToHost(NALUnitLength);
        // Write start code to the elementary stream
        [elementaryStream appendBytes:startCode length:startCodeLength];
        // Write the NAL unit without the AVCC length header to the elementary stream
        [elementaryStream appendBytes:bufferDataPointer + bufferOffset + AVCCHeaderLength
                               length:NALUnitLength];
        // Move to the next NAL unit in the block buffer
        bufferOffset += AVCCHeaderLength + NALUnitLength;
    }
}