Skip to content

vortex-array: Attempt casting inner array to target for extensions#7469

Merged
robert3005 merged 1 commit intovortex-data:developfrom
polarsignals:ext-in-cast
Apr 16, 2026
Merged

vortex-array: Attempt casting inner array to target for extensions#7469
robert3005 merged 1 commit intovortex-data:developfrom
polarsignals:ext-in-cast

Conversation

@brancz
Copy link
Copy Markdown
Contributor

@brancz brancz commented Apr 16, 2026

Summary

Getting

No CastKernel to cast canonical array vortex.ext from vortex.timestamp[ns, tz=UTC](i64) to i64

When running a datafusion query like:

SELECT cast(timestamp as bigint) as timestamp from table;

Where the timestamp array is a utc timestamp type column.

Since the inner array of a timestamp array is int64 already, all we really need to do is try to attempt to cast the inner array of the extension type to the target type. That's exactly what this PR does.

Testing

Added a unit test.

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
Copy link
Copy Markdown
Contributor

@connortsui20 connortsui20 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoops, I must have forgot this!

@connortsui20 connortsui20 added the changelog/fix A bug fix label Apr 16, 2026
@robert3005 robert3005 merged commit d7f07de into vortex-data:develop Apr 16, 2026
111 of 116 checks passed
@brancz brancz deleted the ext-in-cast branch April 16, 2026 14:28
@connortsui20
Copy link
Copy Markdown
Contributor

So just a heads up that we do not know if this behavior will be supported in this exact way in the future as we have not fully figured out the exact semantics of extension type casting. Note that there will definitely be a way to do this cast, but we might enforce that you have to "define" that this cast is allowed somewhere (rather than say that we always allow casting directly from the storage array).

cc @joseph-isaacs

@connortsui20
Copy link
Copy Markdown
Contributor

So we talked offline and we actually do not want this behavior for extension types (we might want to enforce that you cannot cast to or from extension types). Instead, we want to enforce that conversions on extension types have to go through scalar functions like timestamp_to_i64 where the semantics are well defined specifically for that type.

So we are going to revert this, but in order not to break you we will make sure we add functionality so that you can do this operation.

cc @joseph-isaacs @gatesn

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants