Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Field [fieldName] is expected to be type: java.util.List #8

Open
gadaldo opened this issue Oct 25, 2016 · 1 comment
Open

Field [fieldName] is expected to be type: java.util.List #8

gadaldo opened this issue Oct 25, 2016 · 1 comment

Comments

@gadaldo
Copy link

gadaldo commented Oct 25, 2016

I got this exception message when running it on my json string from a TableRow object.
the record is null and the schema type is:

 {
            "name": "customDimensions",
            "type": {
                "type": "array",
                "items": {
                    "type": "record",
                    "namespace": "root",
                    "name": "CustomDimensions",
                    "fields": [
                        {
                            "name": "index",
                            "type": [
                                "long",
                                "null"
                            ]
                        },
                        {
                            "name": "value",
                            "type": [
                                "string",
                                "null"
                            ]
                        }
                    ]
                }
            }
        },
...

The problem is that when I try the export of a Bigquery table with that schema, even if the record is null it works fine and gives me empty array, when I try this in a dataflow, writing the TableRow object in a GenericRecord using this great utility, then it fails.
I think Google internally does the transformation in Proto (as documented here) first and then in Avro so in these steps it probably checks if the value List is null then serialize empty array. I personally applied this check in:
Object read(Schema.Field field, Schema schema, Object value, Deque<String> path, boolean silently)
when case: ARRAY.

Thanks for help anyway.

@adamdubiel
Copy link
Collaborator

@pszymczyk can you take a look at this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants