Battery Shipping Delay Updates
Last Updated July 6, 2022
Most Recent Updates
07-06-2022 - "Individual items contained within the parcel block are continuing to process out of DHS CPS LAX to common carrier. The international parcel block of 800,000 items containing your items clearance process at DHS CPS LAX conclusion has changed to 2022-07-07 5:00 LT (UTC -8) or until further notice. Entirety of parcel block is nearing completion for all items contained. Pickup release notifications for collection of cleared items are being sent to authorized carrier as items clear safety protocols. Scancom servers remain offline at this time for vulnerability mitigation. Please await further notice by manual transmission for any pending DG items."
Representatives for Customs Agents have replied to our numerous requests for updates. DHS agents are expecting to clear all batteries by July 7th due to their backlog and agent staff shortage. They explained Biden’s budget cuts to Dept of Homeland Security has led to 27,000 federal customs agents being fired. These cuts are significantly impacting the speed at which their work is done. Over 3000 DHS agents are no longer present to inspect and examine cargo.
United States Department of Homeland Security is the federal agency in charge of inspecting and clearing the high capacity battery items for safe transit. They check documentation, proof of origin documents and DG safety sheets to ensure that the high capacity battery of this class is safety checked for shippers to handle. It is a federally mandated requirement for this class of battery.
Once the items are X Ray scanned and cleared, a pickup release is given to UPS or FEDEX. At that time the shipper is authorized to collect the items for final delivery. This shipment was expected by us 3 weeks ago and our continued inquiries has resulted in answers to our inquiries.
They also informed us that they are working 24/7 to get through the 83 million backlogged items from the spring season in order to get closer to the normal processing times by August 1 before back to school items are needed and the holiday retail stocking season begins.
We apologize for the circumstances beyond our control which have extended clearance delays and look forward to you receiving the battery at the earliest availability. Shippers will require an additional 6-8 days to transport the items to final destination after Dept of Homeland Security provides authorization to transport these high capacity batteries.
Please know each Triad Battery is equivalent to the power of 500 iPhones. Federal regulators and the shippers take the safety for transport of these items very seriously and there are strict regulations we must pass in order to meet those requirements for the batteries to be delivered.
We are awaiting safe transit clearance notifications to come quickly from US federal authorities in charge.
Thank you for your patience as we continue to proceed with final delivery of your Triad Electric Vehicles order. We are anxiously awaiting your enjoyment of the vehicle upon the arrival of your new 2022 Triad battery.
Some attributing factors to global supply chain disruptions in 2022
Congested US Ports
US Truck driver shortages
2022 winter storms causing backlogs and logistics congestion
Covid cases reducing staff for shipping companies (winter 2022)
Microprocessor delays due to 2020-2021 Covid shutdowns
Early 2022 largest zero day event for computer attacks into servers that process logistics and shipping tracking data.
8 weeks of Covid lockdowns to the world's 3 largest international ports interrupting the flow of goods to 180 countries globally.
Staff shortages in overseas ports and airports within closed loop operations during zero covid policy conditions
Rerouting tens of thousands of commercial cargo flights and ocean vessels to mitigate port closures and staff shortages.
2022 Log4j Server Attack Explained
Log4j Explained - Video
The delayed shipment containing all affected customer battery shipments is currently in the possession of Dept of Homeland Security Customs and Border Patrol (US DHS CBP) agents at CPS (Central Processing Station). The shipment is now in the queue for final inspection and clearance to release all items to the licensed carrier FedEx who will make delivery to the recipient address on file.
We will continue each day to monitor the progress surrounding these important Triad battery shipments and provide updates to this page for your reference. Please be sure to check junk mail and spam folders regularly for direct contact from FedEx shipping notifications once your item is cleared and in transit to your final destination.
Once the Log4j vulnerabilities for international and DG shipments is patched and updated throughout the FedEx Ship Manager system, notifications to recipients by email will process normally.
U.S. Customs is working beyond it's capacity to mitigate, patch and update their system wide vulnerability. They are also working beyond their capacity at this time to clear all all items for release to FedEx for final destination deliveries.
Information regarding shipments will be presented from US Customs agents to the bonded and licensed carrier FedEx only. FedEx will then notify recipients directly by email only. Outbound phone calls by FedEx to recipients for these tracking updates is temporarily suspended at this time.
We sincerely apologize to all our valued clients at Triad Electric Vehicles for this unforeseen and unprecedented delay. We are doing all we can to retrieve real time information and work with the agents in charge to clear these important shipments for all who have been affected at this time.
This is an unforeseen anomalous logistics complication and in no way is considered normal business operations. Battery shipments were in fact sent in advance of vehicle production to account for mandatory safe transit clearance times. Battery shipments have a normal clearance time of 7-10 days at Dept of Homeland Security Customs Border Control. It’s the law under IATA and IACO UN Regulations that they are the governmental regulatory agency who must clear this powerful class of battery to keep delivery drivers and consumers safe while items are transit according to DOT, FAA and US Labor laws.
FedEx is the licensed carrier who makes delivery of this class of safety cleared battery shipment. Once cleared by DHS then it gets released by them and FedEx is notified to collect for pickup, scan the item for tracking purposes and then the item immediately begins the final destination transit process which has a normal time frame of 3-4 days to the door of the receiver. Battery shipment total normal timeline is 12-14 business days, which is on target with the same arrival time of the primary Triad vehicle shipment.
PLEASE READ THIS ENTIRE STATEMENT
US CYBERSECURITY & INFRASTRUCTURE SECURITY AGENCY (CISA) ISSUES EMERGENCY DIRECTIVE
A malicious, self-propagating program known as “Log4j” has affected millions of data servers across the USA encompassing all US Federal agencies including US Dept of Homeland Security Customs and Border Patrol, FBI, CIA, NSA and the US Postal Service, and huge data companies such as Amazon, Google, Oracle, Microsoft, FedEx, and thousands of other corporations, agencies and organizations.
To be clear, Triad Electric Vehicles data is unaffected is completely secured as we do not store data online or over cloud-based servers specifically out of extreme caution over privacy protections.
However, the Log4j data breach is a global disaster preventing FedEx from presenting tracking information on international shipments as they work to resolve the issue. Both United States Department of Homeland Security Customs and Border Patrol (US DHS CBP) and FedEx must complete their computer system mitigations before they can repair their data systems and resolve the issues to be able to provide electronic communications for tracking details on smaller international parcel sized shipments. Specifically, and generally this refers to Triad battery shipments at this time.
During this time, due to these extraordinary circumstances beyond our control, we unable to collect tracking details for international deliveries of small parcel packages due to Log4j vulnerabilities to USDHS CBP, FedEx, UPS and USPS data and tracking servers.
The items in transit to you remain in transit to you at this time using offline systems right now, however the tracking information and scans normally collected and stored on FedEx and USDHS CBP computer servers is unable to be presented by these companies and government agencies while they mitigate this historically massive vulnerability at a critical time of year for shipping.
Log4j is a program that was embedded into massive Apache servers by hackers and allows bad actors to access and take control of those computer systems remotely for complete and full control of these computer systems. Simply put, this is an unimaginably huge data breach where hackers have exploited a very small vulnerability in software running on millions of computer servers all throughout the entire US data infrastructure system.
It’s been devastating to the U.S. supply chain during this critical time and companies and the U.S. federal government are doing all they can to mitigate the largest zero day attack in modern history.
Tens of thousands of federal and private programmers are scrambling to solve the issue, however once they fix one problem it seems another one is created. These are really big data servers, shared systems owned by Amazon Web Services, Google Cloud, Alibaba Cloud, Oracle Cloud, IBM Cloud and others which are leased by the U.S. Federal Government and other U.S. big companies for data storage used for thousands of data points per second.
The DHL tracking system for Triad electric vehicle deliveries DOES NOT have any indications of being affected. Vehicle delivery tracking remains unaffected. FedEx is the carrier who holds the license for transporting high powered battery shipments after clearance from U.S. DHS CBP agents.
The Fedex and U.S. Customs systems used for battery safe transit clearance has been compromised by Log4j vulnerabilities. https://www.fedex.com/en-us/shipping/ship-manager/software.html#tab-4
Triad is calling DHS CBP representatives and FedEx on your behalf each day for any information they can provide, which may include details about their data security patches, offline updates about shipments arriving, updates to existing tracking numbers for server groups that have been repaired, additional service delays, or new tracking numbers for shipments in transit at this time that are in transit to final destinations. We will provide updates to this page as soon as FedEx and US DHS CBP Federal Agencies involved in safe transit clearance of batteries (and other small international air parcel packages) are able to respond to our daily requests.
As of June 30, 2022 Triad managers communicating with representatives of DHS CBP have confirmed all items are in possession of the CBP Clearance Agents at LAX at this time; although DHS CBP representatives stopped short of providing an exact date they will complete their releases for pickup notices to FedEx in order to collect all items in their possession at this time.
FedEx Ship Manager v.3509 Log4j – Service Bulletin Board Message
Due to the urgency of this issue, FedEx recommends upgrading immediately to version 3509 to consume the Apache Log4j 2.16.0 release to address the vulnerabilities for Java 8 users.
FedEx is actively assessing the situation related to the Log4j Remote Code Execution vulnerability.
FedEx Ship Manager™ Software (GSM)
Versions 13.62 and 14.56
Upgrade to version 14.57 upon release in January 2022
The above fixes will remediate the known issues associated with the Apache log4j vulnerability; any subsequent issues will be addressed as needed.
What is the risk?
According to Apache, some Log4j versions of this exploit are ranked 10 out of 10, with a score of 10 representing the most extreme vulnerability. This means an unauthenticated remote actor could exploit this vulnerability to take control of an affected system. To learn more about the risks and ranking click here.
We encourage customers to follow security best practices including those recommended by Apache (Apache Log4j Remote Code Execution), in addition to upgrading FedEx solutions as provided. Details are listed below:
FedEx Ship Manager® (FSM) Versions 340x and above
Upgrade to version 3509 to consume the Apache Log4j 2.16.0 release to address the vulnerability for Java 8 users.
For any additional related questions or the most updated information, customers should contact their Customer Technology representative.
FedEx Delays: Customers frustrated
Official Statement from FedEx "We understand our customers’ frustrations and apologize for any inconvenience as we accelerate contingencies to resolve delays caused by significant package volumes, IT vulnerabilities, and an ongoing industry labor shortage. FedEx is committed to providing service to the best of our ability as local conditions allow. We will continue to monitor these situations in an attempt minimize the impact on service."
Government agencies and (including but not limited to) FedEx, USPS, and many other companies are taking their computer systems offline at this time as a precaution in response to the Log4j vulnerability. CISA says the temporary takedown reflects the extreme risk and urgency of patching the flaw.
Homeland Security secretary Alejandro Mayorkas said on Thursday that he is "extraordinarily concerned" about the vulnerability.
More Facts about Log4Shell (CVE-2021-44228) Vulnerability
- Originally reported on over 1 year ago with no investigation by any U.S. Government agency or mitigation by any U.S. company.
- Privately disclosed to the U.S. Apache Software Foundation by China’s Chen Zhaojun of the Alibaba's Cloud Security Team, again November 24th.
- U.S. Apache Software Foundation publicly disclosed to their U.S. clientele including the Federal Government on December 9th
- The issue was not addressed by any Federal or civilian organizations until December 17th, 2021
THE DHS CYBERSECURITY & INFRASTRUCTURE SECURITY AGENCY (CISA) ISSUES EMERGENCY DIRECTIVE REQUIRING FEDERAL AGENCIES TO MITIGATE APACHE LOG4J VULNERABILITIES
The Department of Homeland Security's Cybersecurity and Infrastructure Security Agency issued an emergency directive on Friday December 17th, 2021 that all federal and federal civilian agencies must assess their systems for mitigations related to the Log4j vulnerability by December 23, 2021.
CISA has determined that this vulnerability poses an unacceptable risk to Federal Civilian Executive Branch agencies and requires emergency action.
This emergency action is based on: (1) the current exploitation of these vulnerabilities by threat actors in external network environments, (2) the likelihood of the vulnerabilities being exploited, (3) the prevalence of the affected software in the federal enterprise, (4) the high potential for a compromise of agency information systems, and (5) the potential impact of a successful compromise.
Official USPS statement: “Computer networks are constantly under attack from criminals who try to exploit vulnerabilities to illegally obtain information. Similar to other companies, the Postal Service’s Information Security program and the Inspection Service uses industry best practices to constantly monitor our network for suspicious activity. Out of an abundance of caution, the Postal Service is further investigating to ensure that anyone who may have sought to access our systems inappropriately is pursued to the fullest extent of the law.”
As a valued customer of Triad Electric Vehicles, you can be assured that we will continue to contact our bonded shipping agents at origin and our FedEx agents here domestically on your behalf each day for manual updates.
Once the release is provided by DHS manually to FedEx as servers are offline, you will immediately be informed by email. FedEx managers have manually entered auto updates to receivers for their shipment(s) in transit at this time.
**NOTE: Because of the Log4J server issue affecting U.S. Customs and FedEx tracking scan databases, it is entirely possible that battery shipments may arrive unannounced without tracking because they are using offline systems at this time. FedEx is changing tracking numbers throughout the delivery process with handheld label printers at pickup and transfer stations without notifying us of tracking number changes. FedEx is currently utilizing smaller node intranet systems while their larger internet systems remain offline as they mitigate the Log4jShell vulnerability.
If items do arrive unannounced without tracking at this time, please let us know by sending a brief email with the final tracking number or a photo of the outer box packaging so we can cross reference this information with our FedEx battery shipping agents who are working hard during this time on your behalf to remain in contact with the U.S. Customs agents who clear all battery shipments for safe transit.
Federal Agents must by law clear all high powered battery shipments for safe transit before carriers can collect items for final delivery. More resources about the compounded delivery delays affecting FedEx and U.S. Customs clearance processes at this time is below.
Additional information affecting possible delays of battery shipments at this time.
Recent Updates about the container ship backlog at U.S ports
Inadequate CBP Staffing
DHS expands paid program to hackers to hunt down Apache vulnerability
Commercial airlines are commonly used by shipping companies to transport goods throughout the U.S supply chain.
Airlines are struggling with cancellations and delays due to severe weather and staffing shortages caused by the omicron variant.
US airlines say the disruption is due to crews testing positive or isolating.
FedEx Corporation FDX is taking a hit from Omicron-induced spike in coronavirus cases and adverse weather conditions across the United States. https://www.yahoo.com/now/fedex-fdx-warns-delays-amid-160004030.html