You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm not sure what the correct way to add support for it would be. Probably another rule but I don't know what it should be called, because the names are so similar. pytestpytest_py_testpy_test_pytest all seem pretty funny. Or maybe it could be an option for py_test or a global one in WORKSPACE.
Reacted by Junyong Lee, bhack, Sergey Kolosovski, Martin Liu, Jonathon Belotti, UebelAndre, Jasha Sommer-Simpson, phlax, Bryce Lampe, Hyojoo Kang / 강효주 and 12 more
If you forget the sys.exit like the person in that mailing list post, Bazel will never know if the tests in that file fail. It would also just be nice if you didn't need 3-4 cryptic lines of boiler plate for every test file.
Reacted by Junyong Lee, Laurenz, peloton-mqiu, Wyatt Anderson, Sergey Kolosovski, Jasha Sommer-Simpson, Maximilian Roos, Chan Kang, Travis-Taylor, Ofey Chan and 4 more
PTAL at pull request #243. It takes a slightly different approach. In particular, it does not pull dependency on pytest on its own, to give users more freedom (e.g. whether to use requirements.txt, virtualenv or vendor pytest in repository).
Suppose defining your own BUILD rule is the right solution: semantically it would belong to /contrib (though there isn't one), instead of /experimental, which is more for things that might become official in the future.
This issue specifically is a pytest 'problem', in the sense that the framework that does 'a lot' (and maybe too much in a sense). From the viewpoint of the build system, py_test rule would not make any assumption about the existence of pytest or another custom testing framework.
Personally I think having a 10-line main file as @ali5h did is most straightforward and I probably won't even create any custom BUILD rule:
py_test(
name = 'my_target',
srcs = [
...
"//path/to:pytest_main",
] ,
main = "//path/to:pytest_main",
...
)
If the test.py has relative import code like from .module import xx, then bazel test would fail. ModuleNotFoundError: No module named '__main__.module'; '__main__' is not a package
Reacted by jxramos, Jialu Zhu, Allen, Derrick Kondo and njlr
We don't use relative imports anywhere in our Bazel workspace. They're not worth the trouble imo, and I wouldn't be surprised at all if implementation details in Python rules made them not work correctly.
@doubler We used to use that imports attr but I think it's a code-smell. For our use cases we've been able to move to absolute imports and remove all use of imports = [] in our BUILD files.
@dayfine makes a solid point, py_test strictly operates off return codes and the absence of segfaults or something like that taking place during test execution. That the phonetics are identical to pytest makes the concepts crowd into each other somewhat unavoidably but they are very distinct.
There's also the more generalized approach from what @groodt shares above, that includes the passing of arguments from bazel py_test targets into pytest via the args attribute. I've used that with great success rendering additional debug detail on failures and even to factor out deep dependency graph test cases lumped into a parametrized pytest case such that each parameter value becomes its own py_test target in the BUILD file via a list comp. Where pytest would normally iterate over all test cases (even those that aren't implicated in the the current diff to the repo) bazel is more nimble to only run those test parameters that are implicated by a particular change. That saves a great deal of test execution runtime by leaning on the remote cache for test results.
importpytestimportsys
...
if__name__=="__main__":
# if using 'bazel test ...'args=sys.argv[1:]
ret_code=pytest.main([__file__, "--verbose"] +args)
# Print relevant debug logs or other artifacts for CI job to have richer contextifret_code!=0:
print_debug_stuff(args)
# Exit with return codesys.exit(ret_code)
Reacted by Jonathon Belotti, Dennis Hoer and Boris Verkhovskiy
Actually now we're hitting the case that @double mentioned above...
If the test.py has relative import code like from .module import xx, then bazel test would fail. ModuleNotFoundError: No module named '__main__.module'; '__main__' is not a package
The use case hoping to have supported for us is to be able to execute test runs from either the bazel process as well as the pytest process. Being able to run from pytest is a convenience for development with respect to debugging and site-packages package development work. It can also be achieved through bazel but one has to do all the folder path hunting and PYTHONPATH setup around the bazel cache I believe.
With pytest I can drop pdb breakpoints, but from bazel something forces the debug session to quit
importpdbpdb.set_trace()
(Pdb)
Traceback (most recent call last):
File ...
File "/usr/lib/python3.6/bdb.py", line 51, in trace_dispatch
return self.dispatch_line(frame)
File "/usr/lib/python3.6/bdb.py", line 70, in dispatch_line
if self.quitting: raise BdbQuit
bdb.BdbQuit
@jxramos RE the "forces the debug session to quit" I've copied below a workaround:
... Unfortunately, this does not work so simply for bazel test and py_test targets. If you add a breakpoint using pdb and run a test that will trigger that breakpoint, you won't be dropped into a REPL the process will crash. As a workaround, change your py_test target to py_binary and do bazel run on it instead of bazel test. Your test code will run and you’ll be dropped into the pdb debugger:
You should be able "bazel run" a py_test target; no need to change it to
py_binary first.
śr., 31 mar 2021 o 01:23 Jonathon Belotti ***@***.***>
napisał(a):
@jxramos <https://github.com/jxramos> RE the "forces the debug session to
quite" I've copied below a workaround:
... Unfortunately, this does not work so simply for bazel test and py_test
targets. If you add a breakpoint using pdb and run a test that will
trigger that breakpoint, you won't be dropped into a REPL the process will
crash. As a workaround, change your py_test target to py_binary and do bazel
run on it instead of bazel test. Your test code will run and you’ll be
dropped into the pdb debugger:
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#240 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKU4J4LJNAB4HUJPG53LTKLTGJMNVANCNFSM4I7SLRTQ>
.
This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!
This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!
https://github.com/caseyduquettesc/rules_python_pytest this came up in the slack yesterday, leaving it here for posterity. I note with some amusement it's almost line-for-line what I invented for my repo and what we're now using at Abnormal.
Maybe you could do the same dance as aignas@6b54226, but sideloading in requirements alongside user code with potential for requirements conflicts seems like trouble. It'd be better to find a way to configure the Python toolchain with a requirement function/macro but dunno if we can do that.
Activity
groodt commentedon Oct 10, 2019
It is already possible to use
pytestwithpy_test.test.py
verhovsky commentedon Oct 11, 2019
Thanks for the quick reply. I saw someone mention this trick on the mailing list https://groups.google.com/forum/?nomobile=true#!searchin/bazel-discuss/pytest|sort:date/bazel-discuss/d1RKUmyve_Q/hm_Dc3XYCAAJ
If you forget the
sys.exitlike the person in that mailing list post, Bazel will never know if the tests in that file fail. It would also just be nice if you didn't need 3-4 cryptic lines of boiler plate for every test file.pstradomski commentedon Oct 11, 2019
@brandjon: it's something I have in my fork of rules_python, using a genrule to create a snippet similar to the one in @groodt comment above.
Do you think it makes sense to add it to experimental/ ?
ali5h commentedon Oct 20, 2019
i added a sample macro for pytest here https://github.com/ali5h/rules_pip/blob/master/defs.bzl#L116
pstradomski commentedon Oct 21, 2019
PTAL at pull request #243. It takes a slightly different approach. In particular, it does not pull dependency on pytest on its own, to give users more freedom (e.g. whether to use requirements.txt, virtualenv or vendor pytest in repository).
ali5h commentedon Oct 21, 2019
i added that as a sample. there is no need to load the
pytestexplicitly, updated the sample to show thisdayfine commentedon Mar 11, 2020
Suppose defining your own BUILD rule is the right solution: semantically it would belong to /contrib (though there isn't one), instead of /experimental, which is more for things that might become official in the future.
This issue specifically is a
pytest'problem', in the sense that the framework that does 'a lot' (and maybe too much in a sense). From the viewpoint of the build system,py_testrule would not make any assumption about the existence ofpytestor another custom testing framework.Personally I think having a 10-line main file as @ali5h did is most straightforward and I probably won't even create any custom BUILD rule:
doubler commentedon Jun 4, 2020
If the
test.pyhas relative import code likefrom .module import xx, then bazel test would fail.ModuleNotFoundError: No module named '__main__.module'; '__main__' is not a packagethundergolfer commentedon Jun 4, 2020
We don't use relative imports anywhere in our Bazel workspace. They're not worth the trouble imo, and I wouldn't be surprised at all if implementation details in Python rules made them not work correctly.
doubler commentedon Jun 8, 2020
@thundergolfer why do python rules has the
imports. https://docs.bazel.build/versions/master/be/python.htmlthundergolfer commentedon Jun 15, 2020
@doubler We used to use that
importsattr but I think it's a code-smell. For our use cases we've been able to move to absolute imports and remove all use ofimports = []in our BUILD files.jxramos commentedon Jul 30, 2020
@dayfine makes a solid point,
py_teststrictly operates off return codes and the absence of segfaults or something like that taking place during test execution. That the phonetics are identical topytestmakes the concepts crowd into each other somewhat unavoidably but they are very distinct.There's also the more generalized approach from what @groodt shares above, that includes the passing of arguments from bazel
py_testtargets into pytest via theargsattribute. I've used that with great success rendering additional debug detail on failures and even to factor out deep dependency graph test cases lumped into a parametrized pytest case such that each parameter value becomes its ownpy_testtarget in the BUILD file via a list comp. Where pytest would normally iterate over all test cases (even those that aren't implicated in the the current diff to the repo) bazel is more nimble to only run those test parameters that are implicated by a particular change. That saves a great deal of test execution runtime by leaning on the remote cache for test results.BUILDtest_foo.pyjxramos commentedon Nov 24, 2020
Actually now we're hitting the case that @double mentioned above...
The use case hoping to have supported for us is to be able to execute test runs from either the
bazelprocess as well as thepytestprocess. Being able to run from pytest is a convenience for development with respect to debugging and site-packages package development work. It can also be achieved through bazel but one has to do all the folder path hunting andPYTHONPATHsetup around the bazel cache I believe.With pytest I can drop
pdbbreakpoints, but from bazel something forces the debug session to quitthundergolfer commentedon Mar 30, 2021
@jxramos RE the "forces the debug session to quit" I've copied below a workaround:
pstradomski commentedon Apr 1, 2021
9 remaining items
github-actions commentedon Jul 17, 2022
This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!
verhovsky commentedon Jul 18, 2022
*activities*
betaboon commentedon Aug 9, 2022
this is the solution I'm using currently: https://gist.github.com/betaboon/c1dd785b5ba468b4df4e382eafff969a
github-actions commentedon Feb 5, 2023
This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!
arrdem commentedon Feb 6, 2023
Should close this out; see #723 (comment)
Ftr here's the solution I've been using happily for some time -
chrislovecnm commentedon Jul 11, 2023
Closing. @arrdem if you want to do a PR with the support I would love to take a look at it.
arrdem commentedon Jul 11, 2023
https://github.com/caseyduquettesc/rules_python_pytest this came up in the slack yesterday, leaving it here for posterity. I note with some amusement it's almost line-for-line what I invented for my repo and what we're now using at Abnormal.
I think this comment https://github.com/caseyduquettesc/rules_python_pytest/blob/main/python_pytest/defs.bzl#L34 captures why having "in the box" support would be hard.
Maybe you could do the same dance as aignas@6b54226, but sideloading in requirements alongside user code with potential for requirements conflicts seems like trouble. It'd be better to find a way to configure the Python toolchain with a
requirementfunction/macro but dunno if we can do that.alexeagle commentedon Jul 31, 2023
Another 3p pointer: https://docs.aspect.build/rules/aspect_rules_py/docs/rules#py_pytest_main provides missing glue for pytest, thanks to @mattem and @f0rmiga
aignas commentedon Sep 27, 2024
Another project that provides
pytestintegration is here:https://github.com/aignas/pytest-bazel
See the doc links in the repo for usage.