Update Has Just Broken Some AWS Glue Studio Scripts

Revert to the classic script engine to fix

A new update to AWS Glue Studio appears to have broken some scripts. If you have edited a previously-working script and it has just broken for no obvious reason, you may be affected. It looks like AWS have updated their script-generation engine for Glue Studio, which is causing some existing Glue jobs to break.

An example of a failed job in Glue Studio caused by this issue

To make matters worse, the errors aren’t very helpful. The console shows ‘SyntaxError: invalid syntax’ and the Cloudwatch Logs error log shows this:

2021-08-05 12:08:36,917 ERROR [main] glue.ProcessLauncher (Logging.scala:logError(70)): Unknown error from Python: Error Traceback is not available.

Some more digging through the logs and you might see this in the job output log:

code = compile(f.read(), fname, 'exec')
File "/tmp/analytics_existing_au_s3_v2.py", line 1003
2WeeksAggregate_node1622537904075 = MyTransform4(glueContext, DynamicFrameCollection({"Prtranscache_node1622532799094": Prtranscache_node1622532799094}, glueContext))
SyntaxError: invalid syntax
During handling of the above exception, another exception occurred:Traceback (most recent call last):
File "/opt/amazon/bin/runscript.py", line 100, in <module>
while "runpy.py" in new_stack.tb_frame.f_code.co_filename:
AttributeError: 'NoneType' object has no attribute 'tb_frame'

The solution is to revert back to the old code generation engine. This can be done by choosing Generate classic script inside the Script tab of the Glue Studio job (see below).

This issue can be resolved by choosing ‘Generate classic script’ to revert to the legacy script engine

Since this feature appears to be brand new and is undocumented I thought I’d share the solution here.

Enterprise cloud, analytics and ML. Computer Science and MBA educated. Triple AWS certified. Scottish but living in Australia.