Real Takes Cue from Broadcasters
RealNetworks New Streaming Facility Is Built
With Efficiencies Typically Found in Radio and Television
by Craig Johnston
When RealNetworks decided to rebuild its Broadcast
Operations Center in downtown Seattle recently, it had a number
of objectives in mind.
Because the customers it aimed to serve with
its streaming and signal distribution would include the biggest
names in radio and television networks, Real’s quality had
to be first cabin.
Particular
"These customers are very picky and they
let you know in a loud way if something isn’t right,"
said Real’s Director of Broadcasting Mark Warner.
Because it stands to prosper as more companies
stream their content, Real wanted to build a showcase facility
to let others see the business prospects in streaming operations.
"We had to show how the technology can
be used to make a positive ROI," said Nagesh Pabbisetty,
vice president and general manager for Real Broadcast Network,
referring to return on investment.
Real wanted to build an efficient operation
that could reliably deliver the product – a streamed signal
– on time, all the time, whether it was serving a 24-by-7
customer or a one-time-only event.
Jim Kreyenhagen, director of marketing for RBN,
said, "We adopted a true ‘broadcaster’ mentality."
Real wanted it to fit an operational and capital
budget so they could earn a return on their own investment.
Real’s original operations center had been
built on an as-needed basis, largely from consumer-grade equipment.
The quality of the equipment was one issue; another was the manpower
it took to make it work.
"They had to get up, tune in the receiver
or connect the phone coupler in one location," said Ben Wolk,
account manager for DST Systems Inc., an information processing
and computer software services provider, which did the system
integration on Real’s upgrade.
"They had to patch in one location and
they had to do the audio processing in another location,"
Wolk said.
"Then they’d come back into their
operations center and actually do the stream. There were four
points of contact for the signal for one operator to get one stream
out."
A primary project aim was to bring all of those
control functions to a single workstation console.
With the upgrade and its centralization of control,
a two-operator shift can control more than 100 events.
Couplers problem
Video comes primarily via satellite, as does
some audio. The building rooftop hosts a dozen satellite dishes
that feed 30 DX Antenna satellite receivers. The antennas and
receivers are controllable through the operators’ consoles
in BOC.
For audio-only projects, the signal also can
come via phone lines and over ISDN. Additionally, the customer
can encode the audio into a stream at its studios and send that
stream to Real via frame relay or ISDN.
The phone couplers represented a particular
challenge. Because phone levels can come in at different levels
and the phone couplers used had an auto-detect and hang-up feature,
the phone system on the outside could trigger the coupler to hang
up.
"We had to work with the phone coupler
manufacture to modify those couplers to make them the most reliable
possible," said Wolk.
Except for signals that enter the building already
stream-encoded, each audio signal is put through a Symetrix AGC
leveler. The Symetrix units automated the level-setting process,
a critical function because the levels coming from the phone couplers
vary so widely.
Centerpiece
The heart of the system is a Philips Venus audio
and video routing switcher. It is configured to supply 96 inputs
and 128 outputs, but can be expanded to 256 by 256. Router functions
are handled automatically through the operator workstation consoles.
Along with the functionality and ergonomics
of the new BOC facility came esthetic features. While the original
room was viewable through glass from a hallway, it consisted of
a large room full of equipment racks with a few operators seated
at computers.
A passerby now finds an operations center complete
with a monitor wall and plasma screens reminiscent of a television
master control room.
Radio demand
Real had anticipated installing the video portion
of the upgrade first, but a flurry of audio-only business required
that section to be built in a hurry.
"The timeframe is probably the biggest
challenge we faced," said DST’s Wolk. "We started
out with the large project as the primary one, then we jumped
into a Phase II project that kind of took precedence – the
audio-only portion, which was the phone couplers with the audio
processing and the patching."
DST had planned to build the racks off-site,
but the rush to complete the audio section changed its plans for
that portion of the project.
"Within a matter of two weeks, from ‘go’
to on-air, it only took us four days to integrate the equipment
and deploy it within their facility," Wolk said. "We
went to the manufacturer, Symetrix, up in Lynnwood (a Seattle
suburb), grabbed all the processors and wired that portion on-site."
Seamless installation
The rest of the upgrade was built and tested
off-site and installed in the Real facility without taking the
existing operation offline.
"I think at the end of the day, the efficiencies
and quality of product that Real puts out nobody can touch,"
said Mark Siegel, manager of DST’s Seattle office.
The second half of Real Broadcast Network is
the network itself. As a one-stop-shop, RBN aims of offer seamless
service from the time the signal enters the building until it’s
delivered via the Internet to the viewer or listener’s computer.
RBN’s Network Operations Center strives
to bring the streamed signal to the customer using the least amount
of the actual Internet possible.
That’s because Internet routing can be
circuitous, possibly zigzagging across the country or even the
globe. This can take the stream through unnecessary switching,
which can result in slowing and lost packets.
To avoid using the Internet to deliver the signal
across the country, RBN sends the signal via private network or
networks to the "edge," a location close to the end
user.
The initial link a customer clicks contacts
a redirecting server at the RBN Seattle facility. Using a number
of factors to determine the customer’s location, that server
directs the customer’s computer to a server in his area.
"Everyone has their own software models
which tell where to get the signal," said Real’s Kreyenhagen,
referring to it as their "secret sauce."
Capacity
RBN normally runs its network at one-third capacity,
which leaves enough cushion to handle huge spikes, like the during
election or Seattle’s earthquake. By incorporating partners
with their own networks, it quickly can scale up to handle increased
demand of a stream.
RBN assigns a program manager to each project.
On a complex project, the program manager enters the process after
the conceptual sale, before a final price is established.
"The customer doesn’t always know
everything involved," said Kreyenhagen. "The program
manager becomes the customer’s champion, his advocate in
the process."
Program managers stay attached to a project
through its completion and are contacted automatically at any
time, day or night, should a problem with the stream occur.
"RBN brought a true ‘broadcast’
mindset to the IP world," said Real’s Pabbisetty. A
team of about 60 people was built from specialists in networking,
software and broadcast.
It takes all three
"People can have expertise in two of those
areas, but no one has all three," said Pabbisetty. He said
he came to Real with two: networking and software.
"I have immersed myself in broadcasting,
learning from the experts we have hired. We all have to understand
the part we don’t know in order to make RBN work."
Things are humming at RBN today. Projects range
from advertiser-supported radio, where Real can replace the spots
from the terrestrial signal with Internet-only commercials, to
the subscription-based Major League Baseball and NBA game feeds,
incorporated in Real’s Gold Pass subscription service.
Real says it has encountered strong demand from
the corporate/enterprise marketplace. Companies have found that
streaming their financial announcements simultaneously to interested
parties around the world is a good way to satisfy new Securities
and Exchange Commission requirements for such disclosures.
Real sees the expertise gained from these various
projects as a magnet for more business.
Craig Johnston is an Internet and multimedia
producer in Seattle.
