Complete doc string examples for functions.py#1435
Conversation
ntjohnson1
force-pushed
the
nick/polish_docstrings
branch
from
021597f to
6a5991c
Compare
python/datafusion/functions.py
Outdated
| ... dfn.functions.alias( | ||
| ... dfn.col("a"), "b", metadata={"info": "test"} | ||
| ... ) | ||
| ... ).collect_column("b")[0].as_py() | ||
| 1 |
There was a problem hiding this comment.
This doesn't really demonstrate the metadata output. Instead of converting to py maybe we need to check something else. I'm not 100% sure what right now but I'll take a look after finishing this review.
There was a problem hiding this comment.
df.select(
dfn.functions.alias(
dfn.col("a"), "b", metadata={"info": "test"}
)
).schema()
This does show the metadata. Converting to pyarrow scalar appears to drop it. Converting to a pyarrow table does keep the metadata on the table, but then selecting the appropriate array still you can't see the metadata. :|
There was a problem hiding this comment.
I printed out the schema instead
python/datafusion/functions.py
Outdated
| ... dfn.col("a"), dfn.lit(10), dfn.lit(".") | ||
| ... ).alias("lpad")) |
There was a problem hiding this comment.
I'm wondering if we should use keywords to help make it more explicit, but not a blocker.
There was a problem hiding this comment.
Added keyword for optional args here
| >>> ctx = dfn.SessionContext() | ||
| >>> df = ctx.from_pydict({"a": [1, 2, 3]}) | ||
| >>> result = df.aggregate([], [dfn.functions.count(dfn.col("a")).alias("v")]) | ||
| >>> result.collect_column("v")[0].as_py() | ||
| 3 | ||
|
|
||
| >>> df = ctx.from_pydict({"a": [1, 1, 2, 3]}) | ||
| >>> result = df.aggregate( | ||
| ... [], [dfn.functions.count( | ||
| ... dfn.col("a"), distinct=True, | ||
| ... filter=dfn.col("a") > dfn.lit(1), | ||
| ... ).alias("v")]) | ||
| >>> result.collect_column("v")[0].as_py() | ||
| 2 |
There was a problem hiding this comment.
nit: would be slightly easier for users to see the difference in the examples if they had the same formatting but I bet it's the linter making the call
There was a problem hiding this comment.
The linter is a little odd inside the docstrings so it's semi linter supported. I did a quick pass to make the formatting more similar throughout the multi example situations.
2 participants
Which issue does this PR close?
Closes #1434 and #1433
Rationale for this change
Added coverage over MOST functions in functions.py with examples but split things up to be reviewable. There were a couple global notes that got lost across the many PRs
This relies on #1418 landing first since I branched from there.(MERGED and rebased)This PR isn't as large as the diff suggests because a lot of changes are simple formatting. Reviewing by commit should make it much easier.