Archive
Processor Trouble?
Received complaints from PD about some odd changes on air in the sound of the station. Last time this happened I basically pulled the AES digital input briefly. Once it was back in all was stable again. Thought nothing of it as little blips and burps occur in the electronics world.
Today there was another complaint. OK, a couple months have passed, we had some fun, let’s check things out. My first indication of a possible issue on the Omnia 11 is the fact that when I look at the detail page of the input settings I saw AES Sync: Unlocked. Now that is very interesting. I run a SAS shop, so my reference for anything digital is this system, and if one box is having issues, then all units would have issues. I spot checked anyways. All 48kHz sample rate, all good. Let’s do the old unplug the AES input from the processor, wait a moment, and plug it back in. No, still unlocked. time to dig.
I look at the output of the processor to see what it should be synchronized with and find that it was set to External. No, that should be set to Input as I am using the SAS as my house clock. Changed it. Still AES unlocked. Let’s plug a source into the External Reference on the processor. Still unlocked. Now this has become interesting, so back to the output and change it to sync with the internal at 48k. AES unlocked. I believe we are due for a reboot, so I check online for any updates. I see there is a 3.6 for the Omnia 11, so I get it. In addition, I send a note to the Alliance. They confirmed the unit either needs a reboot, or there is an I/O issue with it. Tomorrow I get to test this theory.
A new day and morning drive is over. Let’s play processor roulette! I double check and make sure I still the AES sync still unlocked. I have a spare Orban 8500 setup and ready to go, so let’s do this. Uploading the file is a breeze. As the first thing to do after the update is applies is a power cycle, everything gets a fresh start. Mental note I saw that the I/O board received and update during this cycle.
After a few minutes, like almost 10, the processor is up and running. Only details is to apply the saved I/O configuration and preset. First thing I check is AES sync. It reports locked. I change the setting to sync to input. It stays locked. I’m good to go. I figure I have another 11 I proceeded to update that one, too. Only time will tell if this is permanent. Being a computer, it does not surprise me that a reboot would take care of the issue. I will continue to monitor this and keep an ear out for screaming PDs with audio degradation!
Cheers!
When Old and New Don’t Mix
A few months back the decision was made to “rush” a MaxxCasting system. The company teamed up with GEO Broadcasting and GatesAir to have this system installed. Currently in analog mode it works fairly well. When the baseball season finally ends, we become the test for doing HD as well. Check it out, it is interesting. The part I have issues is making the old, a 2006 Harris (GatesAir) HTHD+, and the new, a Flexiva FAX50 exciter, play nice.
I knew I would have issues from day one with this installation. Having no involvement other than being told what needs to be done, providing input was limited. At the onset I informed everyone on this project by interjecting a simple question: Will there be any issues utilizing our old HTHD+ transmitter for this? An early 2006 edition of the HTHD+ transmitter as we were “forced” into the early adoption of HD radio, this transmitter installation was a rush job from the get go. As an amplifier I have never had any issues with the transmitter, but I have had issues with exciters. We had 3 different Flexstar exciters in this thing, yet I have only replaced the tube 4 times. Anyways, the answer to the question was basically, yes, don’t worry about it.
Without being involved other than do as you are told routine, the day came to install the FAX50 as an exciter. Without a full rundown of the Geo-synchronization of the whole system, let it be known the Flexstar was not capable, thus the swap. I has the exciter out. I had the filler plate. I had cable adapter kit. Whoa, hold on. The kit labeled for a Z series transmitter? I told everyone this was an HTHD+. Hmmmm? Let’s proceed with getting the new exciter installed in the transmitter. After the first indicator of an issue (the cable), the second indicator of more fun to come was the fact that the exciter did NOT go into the transmitter! I was told otherwise, and I was provided with no instructions. Phone call time. Sure enough, the interface kit was wrong, and no, the FAX does not fit inside the transmitter, but must be externally mounted. New interface cable kit and instructions were on their way.
The new cable kit arrived with the new instructions. Now things began to make sense. I cleared out rack space and place the exciter in a location that was just accessible to comfortably been seen, i.e. not at the top of the rack, but at least a quarter way down. That was the maximum length of the new cables. Installation complete. Testing into the station load I got to learn how to setup the exciter as it did not come setup for our station. It was setup on the wrong frequency and as a transmitter, full 50 watts, not as an exciter of which we run about 4 watts. Details of this for another time. Rest assured it was not too bad and I had the transmitter back on the air soon enough.
Mixing the old and new. As we all know, the HD E2X, or Exporter to Exgine, stream will have packet drops on occasion or some other ISP induced oddity. In the case of anything somewhat prolonged or unusual, the exciters are designed to shut the HD off, suppress the carriers, and the station goes analog only for that brief time. When the E2X stream returns, the exciter detects it and HD carriers return. All good. Well, the FAX series exciters are some fancy boxes and are designed as stand alone transmitter. These range in power from the 50 watts to high power. As such when the E2X is gone, FAX is designed to drop power level and normalize to analog operation. When the E2X stream returns, the FAX adjusts, the power level changes to accommodate the HD carriers. All good when in an all inclusive box, when driving an external transmitter it causes some serous output power swings, and at times even a brief no power out. Our transmitter is licensed for 18kW TPO. I have witnessed this drop to zero and shoot to about 20kW all within a few seconds. I had four occurrences just this morning of lower power alarms followed by high power alarms. The APC and drive level required drops suddenly during this incidences that the transmitter PA cannot handle it. I wonder what stress this will place on my recently replaced tube?
GatesAir is aware of this and is considering options. I have suggested to the company they may wish to just upgrade the transmitter. We will see. I think the lesson that reinforces my philosophy is not to attempt a new project which mixes technology without considering all the pieces, and be prepared to invest in replacing the old even if it is going to cost you. Consistency is key to a smoothly running plant.
Cheers!
WideOrbit Automation & EAS
When we made our move into new facilities, we had to change our automation/playback system from NexGen to WideOrbit. At first we thought that it may be a good change. Maybe even take care of or streamlining some of the processes. The plan was to continue with an IP based facility, so we needed to make sure there was not falling back to a ton of cabling and physical GPIO for this. Well, a year+ later and we continue to fight how to reliably and consistently run EAS tests from WideOrbit.
As we saw this was turning into and issue, we setup our Sage Endec to send a RWT on Sunday mornings while we worked out the details of how WideOrbit deals with them. This has turned out to be a good thing as various attempts and inconsistencies caused scheduled tests to be missed. We would make them up, or attempt to, as soon as we found out of such events, but some got through. This begged the question why? We stuck one of our guys on it. Week after week, months gone by, there was no rhyme or reason to why some worked and some didn’t. Unfortunately we had cutbacks and the person working on this was released. The void had to be filled and I took up the challenge.
I had, with a little work with NexGen support, the Endec and NexGen working well via Ethernet. Weekly tests were scheduled and they would run. Monthly tests or alerts came in, and NexGen would drop them into the log, fire them, and resume operation. This was 7+ years ago! I believe I even wrote about it in a previous blog. I began to dig in and find out what is so different here that we cannot do the same.
The WideOrbit documents show a log of information if I wired up GPIO, and they do have the EAS widget. I began questioning operators on how they used the system. What did they do? Who did what when it worked? Who did what when it didn’t? Yes, using the widget produced different results. I decided to dig deeper and pulled out the trusty Ethernet tap and placed that in front of the Endec. Data gathering time has begun. I learned a lot of what the Endec sends and what it wants to see. I sent notes to Sage support for clarification. I now know what the heartbeat is. I know what a query from the workstation is and the Endec response. Time to see what happens when they run their tests.
My packet captures were full of fun stuff. One forgets that the Endec, and all EAS units, communicate with the CAPS servers, so all that traffic I knew had to be ignored. Pretty cool though to see the chatter, but time to filter that out. I only cared about the talk between the workstation and the Endec. There could be much, but then I noticed the Endec sending a series of strings consistently timed. The heartbeat. The Endec sends a heartbeat to tell systems the unit is there waiting to play. Included in the heartbeat is the expected delay if a RWT is run. In our case it is 14.45s. I never knew that, so something new is learned. Then I see a packet sent from the workstation and the Endec immediately replied. I learned that this means the EAS widget on the workstation was definitely connected as this was a query to the Endec, “are you there?” The Endec sends a reply code, so all good there. Why are tests not being run?
After much digging, and observations, I still did not know why the inconsistency, so I reached out to our engineering community. With a couple of responses of positive results, I needed to chat. The main thing that came up and was consistent between the two engineers was the fact that a test run via the widget was dropped into the stack. From the information I received from our operators some would drop the test into the playlist/log in advance of the test, while a few would drop it into the stack. I took this information and decided to put it to the test. I went to each of 4 stations control rooms and asked, stack or playlist? The answers were mostly playlist insertion with a couple of stack if it was convenient. Each station we dropped the test into the stack. Each time it triggered. Part one of the mystery may be solved. Instructions went out to the air staffs’ to use the stack insertion.
All this fine if you have someone in the studio to run the tests. These days stations are unattended a good portion of a day to all day. How does a system like this deal with these situations. Turns out WideOrbit really does not have a system when using Ethernet or IP control. If you are wired for GPIO they have some provisions. So, I decided to experiment and see if I can at least fire a RWT using a workflow. I configured a workflow with the instructions to connect to the Endec’s IP address and port, and send the string to run a test. To make it testable, I put it on a hotkey so I can fire it at any time. The moment of truth, I click on the hotkey. I check out the Endec. Nothing. I do it a couple more times. Nothing. I create a telnet session from the workstation to the Endec and send the command. Works like a champ. I check the WideOrbit logs and they claim the workflow ran and sent the string, but the Endec did not respond nor did the packet get captured, but he telnet did.
At this time I am at a loss on how to make a completely working IP control of the Endec using WideOrbit. In today’s time and the proliferation of unattended stations one would think this would be of high importance. When asked, WideOrbit support responds with run the Endec in automatic and let it run. I hate to say it, but when you are in a top 20 market the last thing the PD, GM, and sales manager wants to hear is an EAS test stepping all over a commercial break or song. Sure it can be done with GPIO, but to me that is an 80’s, last resort solution. Leave a comment if you have any thoughts.
Cheers!
Should I or Should I Not
So, I’ve been a been absent on posting. I began to start wondering how helpful my blog posts are. Leave a comment if you think these are worth it.
Question: Do you want to see more posts?
Question: What subject matter, or should I post on what I am doing?
I also dropped social media due to the inundation of junk that was not worth weeding through. A shame that people cannot use tools properly and effectively. In any case, would a twitter feed by useful again?
Cheers!
Bill
Happy Holidays!
A long year is coming to a close. Here we were involved in the most complex and annoying facilities move ever. As I gather notes, I think I will post some highlights of the move. I am also going to take the age old advise and say, if you have nothing good to say about someone, do not say anything at all.
Have a Happy Holiday season and best wishes for the New Year!
AoIP: Dante by Audinate
Love it. Simple. Just build your network properly.
We are what I would say is 3/4 the way through getting these facilities in a psuedo-complete state. The one thing that I think has been the most reliable is our audio network, Dante AoIP. Simple, efficient, and easy to maintain. I need to do a count, but I am sure I have over 50 devices on this network/system. I know I am not done.
The devices range for KDL modules in my SAS 32KD frame, to SAS Rio Bravos, and a multitude of Dante Virtual Sound Cards (DVS). Using the Dante Controller application makes it very easy to setup, configure, and maintain. After putting all this fun stuff together I decided to visit the Audinate site and check with their online information. I decided to go through the steps and check out the training. I did all this just this past weekend. The end result: I have a Dante Level 2 certification.
I hope to catch up and spend time on some details on why I chose Dante, and how I implemented it. For now, know that I am a pleased Audinate Dante customer.
NAB Show 2017 Recap
I had to cancel my NAB trip this year!!!!
So there you have it. The recap of the century. What was the excuse to cancel? It was cancel, too. All plans were made: flight, hotel, appointments, dinner. All cancelled. Oh, the excuse: One of our stations decided to change formats!
First we are on the back half of our facilities move. More on that someday. Needless to say, there are many things that are not in place or implemented at this time, so a format change does not make life too easy, especially when one of the staff was on vacation. Well, we did it. Format done, but many important meets were missed.
The life of radio sure makes things interesting and last second changes are among them.
Cheers!
PS: I was involved with putting the first AM all sports station on the air in San Diego. I now can say I as involved with putting the first FM all sports station on the air in San Diego.
Does Your Transmitter Run on 66 Vac?
Friday and my family was in L.A. I was planning on taking the train up to meet with them as it as Grandma’s birthday, but it was contingent on how the week was heading. South, that is where it was heading!
Sitting at the my desk doing a couple of things the the silence alarm goes off. Not just one, but two stations collocated. For that to happen their are only two things that will cause that, main STL to the site or power. I quickly switch one to the our Aux site, the other I poke at and switch STLs. Back on. Off to the site I go. Mind you I get a low power warning, and both transmitters were on at about 3kW each. Each transmitter is a Nautel NV20, the very ones that I have posted about before. In addition a get an high room temperature alarm, so HVAC is down.
I get to the site. Sure enough the TV station’s generator is running. I open my door and greeted with hot air. My lights work, but two of 3 UPSs are beeping away running on battery. Looking at my surge suppressors, two phases are missing! Yes, two, and one NV20 was on. I had turned one off earlier when I switched that station to an Aux site. Out comes the Fluke and I start measuring the phases. One is running fine at 120V, the other two are running 66V. One rack is powered while the other two are on battery. I make provisions to route power to equipment in a backed up rack preparing for the batteries to die. I look at the transmitter running. 3.6kW TPO and the status tells me 2 phases are missing. All I can say is, “wow.” I had a similar incident years ago with a single phase loss, but this time 2!
Once I had my stuff stable, I noticed the power company, SDG&E, was already outside on the street. Ah, they were doing something and it went wrong. Of course the guy in the truck had no update nor did he know where the issue was. The guys at the fire station, following protocol, had moved their trucks out and were talking with another SDG&E person. Eventually 3 trucks were up the street. Within about 45 minutes of being on site, power was restored.
Chalk another one up to Nautel. The power distribution and fold-back on the NV20s worked flawless. Needless to say, my HVAC recovered and all other equipment survived the trauma. Let’s see if the rest of the weekend remains quiet.
Cheers!
Fixin’ Things: Week’s Update
Figured I would try and and get back into the swing of things and update a bit more often. Mostly boring stuff, but, hey, someone has to do it! This week I have a bit of F.O.R.D. action to mention along with plans to repair a Nautel NV20. Also a little fun with NexGen and Sage communications. A little bit of everything.
F.O.R.D. = Fix Or Repair Daily. Harris Z10. This box just sits there and is used on occasion when I do a Nautel update. As it sits, it fails. No, it doesn’t need to be turned on, it just fails. I should clarify, it shows Faults. Currently 4 when I look at the logs, used to be 5. Let me remind you, it gets these just sitting idle. Apparently 2 of the 4, was 3 of 5, are PA modules showing the PS#_OT, or over temperature fault. Again, it isn’t running. A little research and some interaction with tech support a couple months ago, I finally ordered the parts through Mouser and did a test repair. This repair entails the replacement of a capacitor and thermistor. Easy enough except they are surface mount, so I finally got a chance to experience this type of repair first hand. All I can say is it takes patience. Needless to say, the test PA module installed and fault is cleared. I have 2 more to do. The other faults I will act on when I get these obvious ones out of the way. Unfortunately it looks like one of these pending faults corresponds to a dead PA.
But, wait, did I say a Nautel repair is imminent? Yeah, I guess I pushed the old beast a bit too hard as I prepare for an HD carrier increase. I began to adjust the HD PA voltage to accommodate the new power increase and two PA modules said they did not like that, and Poof! I just finished doing some research and I have all the parts to replace them. I even have the tools! What did I find out today? I have version A of the modules and all the latest documents show they are up to C, and they look more modular! The old ones require de-soldering and then replacement. At least it doesn’t look to bad. I also get to perform the modifications that the modules required a few years back to improve efficiency. All in all, I call this fun.
We us IP to control most everything these days, and this includes how the RCS NexGen communicates with the Sage Endec. Sure enough, this week I noticed that the RWT did not fire properly. This is normally caused by the old architecture of NexGen where it just decides to not talk. After a couple of tests I had to do the old CTRL+Alt+Shift+F4 on the A-server so it would load a fresh database. Why it decided to start doing this is beyond me. The other two stations do not have this issue. I know we will be moving off this system some day, so whatever we use better be IP savvy in running all tests and alerts.
Interspersed among all this fun is Microsoft and Windows 10. Thanks. Apparently there is an issue with Win 10 and USB 3.0. At least with some of our stuff, more specifically Digigram and the UAX-220v2. Thought he device uses generic Windows drivers, the system will throw a fault and reboot spontaneously. When trying to track it down, the Event Logs show nothing except the “previous shutdown was unexpected” error. I finally caught a memory dump and analysed it. A reference to a USB device was there. As a test I removed the UAX and that machine has settled down. I think I will need to experiment with a couple of other USB sound devices and see if it is isolated to Digigram or it is more universal issue. Sad if it is the UAX as the beauty of it was plug and play with no special drivers.
If you get a long weekend, enjoy! If not, take advantage of the time you have!
Cheers.
NAB Show 2016 Recap
Ever been so busy that you cannot dig out? Well, I’m finally digging out! I will start out saying that I need more time on the floor and will have to look for ways to weasel more time, but last week it would not have mattered since as the day I returned to work we had a station launch. Welcome the new Sunny 98.1! I could start writing on all the things I didn’t get done from my list, but that would be boring, so let’s dive in and see what I did see physically, and what I feel is the number one trend by many. First the products that stood out to me.
Tieline Technology (www.tieline.com) introduced the ViA. A compact codec with an new improved user interface. Designed around the world of IP connectivity the unit has built-in WiFi, a physical network interface, and USB ports for a customer provided USB modem. If you are familiar with the Report-IT app, then the use of the codec will be intuitive. The codec comes with 3 XLR type inputs and 3 headphone outputs. Location of the headphone jacks are next to their respective inputs making it very convenient for all cable runs. I do see room for improvement, and rest assure I will be in contact with them. I hope to be able to spend a bit more time with this device when they roll them out. It is no secret that I’m a Tieline supporter and have tested and evaluated a number of their devices.
Wheatstone (www.wheatstone.com) has acquired Audion Labs VoxPro. Did you know that? Yeah, I did. VoxPro is a very strong and well used phone editor and I am glad someone picked up the product, and they appear to be very serious about it. They retained the developer, Rick Bidlack, who seems to be getting the support he needs to move the product forward. VoxPro 5 was released a while back with some improvements, and 5.1 came out just a month or so ago to address some very important issues. As you can tell, with the release of VoxPro 6, they are standing behind the product. The physical controller now has a new paint scheme, but enough of cosmetics, the new UI which came out with 5 is looking good in 6. The ability to detach the hotkeys will make some morning show folks quite happy along with the capabilities to record while playing back. I have not tried that, but we’ll see how the hardware (PC) handles that work load. Finally a little multi-track editing capabilities are there for those who want to get more complex in their work. I have experienced some issues with Windows 8 and 10, and they appear to be OS issues. I feel for software developers that have to deal with such headaches as Microsoft isn’t making it easy for them! I am continuing to watch here as I feel there will be more coming soon.
DEVA Broadcast (www.devabroadcast.com) continues to impress me. Each year at the show I see something new and I just want to try it. Maybe this year I will be able to get my hands on a couple of their devices. I could list a whole slew of devices that are new or upcoming and that would be space consuming, so I will give a mini list of what I liked.
Confidence monitoring, both HD and FM in the DB3011 (HD) and the DB3010 (FM). Both are radio monitors with IP audio streams, so you set them at a site and stream the audio back. Listing what you can do with these devices will take many pages, so hit their website. Noteable is you do it at your computer. You can monitor nearly everything and then some. They think of almost everything.
Audio processors. Did you know DEVA does processing? I did not. On the floor I spent some time listening, via headphones, and I was quite impressed what a single rack unit box can do. Look for the DB6400(4-band) and the DB6000(6-band). Again you can access these via IP and front panel. The details you can get into is amazing.
Finally the DB7007 Re-Broadcast receiver. If you are in need of a receiver for a translator site or even a booster, this may be your choice. It is feature packed and has fail-over to many options like IP stream or even MP3 with a built in SD card slot. In addition, and an issue some stations have, is RDS re-encoding. You can get the proper station calls on that station and even the slogans. Now if we can get one that can re-encode PPM, eh? Again, this unit can be accessed via web interface to monitor and control.
All the devices come at a very good price point. If budget is a concern, you may wish to look at these.
Inovonics (www.inovonicsbroadcast.com) is my go to for the space saving INO series of tuners. Their new 638 is the HD Radio SiteStreamer which is the HD evolution of the 635 FM/RDS SiteStreamer. All accessible via IP and web interface you now know what is happening at the site with all the data. A nice feature is polling. One box will poll your HD, HD2, HD3 in a time frame set by the user all with silence monitoring and alarm notification. Get that, one box. I also like their 650 Arron FM Rebroadcast Receiver. This one introduced the re-encoding of RDS information, and an improved tuner to really pickup that main station if you are way out there or in an area of marginal reception. I acquired an 808 Justin for diveristy delay on one station. The box works well. I have had little time to research on of the other devices out there, but I know they exist. I feel more enhancements are coming to the 638 than what I saw at the show, so place this on the watch list.
While poking around the booths, the one trend that I picked up was monitoring and control. IP accessibility at sites have improved with better land and microwave connectivity. Manufacturers are embracing this. WorldCast (www.worldcastsystems.com) introduced the WorldCast Manager. Burk Technology (www.burk.com) introduced SNMP monitoring in their ARC Plus Touch remote control. Both systems are delving deeper into the SNMP data gathering. Those with an ARC Plus Touch can upgrade to this capability, those with the ARC Plus only can look into their upgrade package deal. What is make the life better for Burk users is the SNMP monitoring is now at the local site and you have channel assignment capabilities. Those of us with the older ARC Plus rely on the AutoPilot software for SNMP monitoring which may be at one location thus requiring an always up data connection. This upgrade now gives local options and redundancy.
Look for more monitoring and control solutions out there. I know there are others doing it, I just did not have a chance to visit them. As operators cut staff and the fact that technology allows for more granular monitoring embracing this will help minimize off-air situations and help with troubleshooting. Know where a system fails based on status points will quickly show where an issue lies. Boxes that can communicate with each other will help expedite switching to alternate audio feeds. If you have not begun to build such a system, you may wish to consider it.
Two days on the floor was not enough for me this year, but I made the best of it. I enjoy catching up with colleagues year after year. I like to see what is new, or in reality, what has improved. New is lacking, but improvements abound, and you can see who is embracing the improvements and incorporating them into their systems. And, if anyone can find a better, rack mounted, HD radio that displays Artist Experience other than the cheap desktop thing, I sure would like to know about it. Now get back to work, y’all!
Cheers!
Tech Notes from the Field 