– Reports of circuit impact into the CityFibre TSC
– CityFibre TSC engaged CityFibre NOC for initial investigations
– CityFibre NOC confirmed an issue seen on active monitoring
– MI process engaged
– MI accepted
– Internal Bridge call scheduled
– NOC investigations ongoing with several examples of affected circuits provided from information gathering by TSC
– Further impact to Consumer circuits discovered and acknowledged
– NOC investigations determined an issue within the core network emanating from a specific location
– NOC contacted hardware supplier and raised a Priority 1 case
– All logs provided to hardware supplier for analysis
– Internal Bridge call convened
– Conference call between CityFibre NOC and hardware supplier convened
– Following discussions between CityFibre NOC and our hardware supplier, there have been developments on this incident in regards to restoration.
– It has been found that the origin point of the issue is on a line card situated within a core network device.
– Soft clear of card performed without success
– Full remote reboot of car performed which was successful for a period of approx. 30 mins before the issue manifested again
– Further internal call held with CityFibre NOC and Hardware Supplier to agree next steps
Current Action Plan:
Escalation made to to Hardware Supplier as no current ETA can be given. Engineering attendance has been requested to CityFibre Slough Datacentre to carry out a reseat of the card. Once reseated they will remain on site for at least 30 mins to confirm success. If the issue manifests again then engineer will proceed with a Card Swap out.