[Reading Notes] Multimedia Systems — Algorithms, Standards and Industry Practices, Chapter 3 Media Representation and Media Formats
1 Digital Image
1.1 Digital Representation of imags
image: width, height, pixel depth
pixel depth: number of bits to represent the pixel, same for all pixels of a given iamge.
Num of bits depends on the color space representation(gray or color) and is typically segregated into channels. Total number is the sum of the number of bits used in each channel.
Gray image: 8 bits for pixel.
Color: RGB: 8 bits each channel, total is 24 bits.
Color: RGBA: alpha, opacity. Total 32 bits.
Number of channel ranges from one to four,
Image to Printing: halftone. Resolution: freq of dots per inch. Color image: four halftone channels are used: cyan(青色), magenta(洋红色), yellow and black(why?)
The 8bit alpha channel(only alpha channel’s meaning) value for each pixel associates the degree of importance of each pixel in a compositing operation. A value of 0 indicates that the pixel will not be entirely composited, 255 means totally composited.
1.2 Aspect ratios
3:2 (print photo)
4:3(tv)
16:9(high-definition image)
47:20(anamorphic format in cinema)
Change image aspect ratio can change the perceived appearence of the pixel sizes. Known as the pixel aspect ratio(PAR).
1.3 Digital Image Formats
raw image file: no compression.
Dithering: add white noise to signal.
2 Digital Video
2. 1 Representation of Digital Video
Two important properties: frame rate and scanning format
Scanning format: outcome of analog tech, can be represented as interlaced or progressive.
2.2 Analog Video and TV
2.2.1 Conversion to YUV
RGB transforms into YUV signal.
Y: intensity info
UV: color info,
humans are more tolerant to color distortions.
2.2.2 Analog Video Scanning
Interlaced scanning format:
frame -> odd filed and even field.
electron gun draws odd lines then even lines.
NTSC(60Hz), 1/60 for each field, 1/30 for both.
occasionally motion artifacts.
Progressive Scanning
line by line
2.3 Types of Video Signals
Composite video
S-video and component video, better quality
Diff: YUV signal and the individual components are setn collectively or seperately
2.3.1 Composite Video
Also named: baseband video or RCA video.
It is the analog wave-form that conveys the image data in the conventional NTSC television signal.Composite video contains both chrominance (color) and luminance (brightness) information, along with synchronization and blanking pulses, all together in a single signal
Pros: reduce bandwidth and achieve real-time transmission.
Cons: interference between chrominace and luminance
2.3.2 S-Video
Super-video also: Y/C video. YUV, UV->C, YC transmit seperately.
Pros: reduce interference
Cons: bandwidth
2.3.3 Component Video
YUV all three seperately.
2.4 YUV Subsampling Schemes
RGB: captured and displayed.
YUV: transmission.
Color could be subsampled because of human eye’s insensitivity.
4:1:1 == 4:2:0 12 bits per pixel
4:2:2 16bits 8 + 8 2 0.5 = 16
2.5 Digital Video Formats
3 Digital Audio
3.1 Digital Representation of Audio
Analog to digital: pulse code modulation(PCM)
sampling rate, quantization.
Channels: one(mono), two(stereo), multi-channel(surround sound)
Channel could equal to number of speaker.
3.2 Surround Sound
5.1 surround sound: 6 speakers.
7.1: add two spatial speakers
3.3 Spatial Audio
Spatial audio makes use of Head Related Transfer(HRTF) and reverberation to simulate the virtual localization of the sound source.
HRTF: describes how the sound is filtered by the diffraction/reflection and other physical phenomena that happen because of the head position/orientation, pinna, and torso before the sound reaches the inner ear for perception.
Reverberation: is the persistence of sound in a particular space after the original sound is removed.
3.4 Commoly used audio formats
4 Graphics
- Vector 2. rasters
vector: Pros: provide infinite resolution Cons: convert to raster image to display.
raster: represented as a grid pixels, each pixel having x,y coordinates and a value that corresponds to a color.
4.1.1 2D vector graphics representations
Soomth primitive or discrete primitive
Discrete primitive: store points such as triangle or polygon.
4.1.2 Animation using 2d graphics
Translation, rotation, scaling,
In order to take use of same operation— multiply, use homogeneous coordinates.
