Implement Bitrix24Partners support (issue #70)#109
Open
KarlsonComplete wants to merge 33 commits into
Conversation
- Replace `docker-compose` with `docker compose` commands in `Makefile` - Remove deprecated `setBitrix24PartnerId` method from `Bitrix24Partner` class
…tory` instance and simplify `setPhone` logic in `Bitrix24Partner`.
…improved context.
…andler создания партнера.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors the Bitrix24Partners module by renaming bitrix24PartnerId to bitrix24PartnerNumber and transitioning the repository to a composition-based approach. It also introduces email validation, updates PHP and PHPUnit version requirements, adds a maintainer skill definition, and improves testing infrastructure with a new partner builder. Feedback identifies inconsistent Docker command usage in the Makefile, suggests a more standard PHP version constraint, notes PSR-12 style regressions, recommends better HTML error handling, and identifies a typo in a directory name.
camaxtly
requested changes
Apr 19, 2026
…cked, MarkAsActive и Update.
…logic: - Add `Bitrix24PartnerNotFoundException` handling to UseCase Handlers (`Create`, `Update`, `Delete`, `MarkAsBlocked`, `MarkAsActive`) for better error management and logging. - Enhance field validation in `Update.Command` to ensure stricter checks. - Introduce `logoUrl` field in `Bitrix24Partner` entity and adjust related UseCases and tests. - Refactor tests to include new scenarios and improve data consistency.
- Replace `docker-compose` with `docker compose` in `Makefile`. - Introduce `bitrix24:partners:scrape:v2` command with updated logic. - Register `ScrapePartnersCommand_V2` in the console application.
- Integrate Monolog for detailed logging throughout the scraping process. - Refactor `ScrapePartnersCommand_V2` to enhance parsing logic and improve error handling. - Introduce support for fetching partner logos and storing results in a CSV file. - Streamline HTTP requests and implement configurable delays.
…`Makefile` and `console` setup.
…а, изменил сигнатуры методов сущности и добавил поддержку логотипа партнера.
KarlsonComplete
changed the title
camaxtly
reviewed
Apr 26, 2026
camaxtly
reviewed
Apr 26, 2026
camaxtly
requested changes
Apr 26, 2026
- Add `Uuid::v7()` support in `Bitrix24Partner` constructors and builders. - Improve validation for partner creation with `bitrix24PartnerNumber`. - Update Doctrine mapping for `bitrix24PartnerNumber` to use `unique` attribute.
…erBuilder: - Remove redundant `withBitrix24PartnerNumber` calls from tests. - Adjust `Bitrix24PartnerBuilder` to expand possible number range. - Reorder `clear()` in `Update.HandlerTest` for better test consistency.
…dize database truncation across functional tests.
- Implement stricter non-empty string checks for `site`, `email`, `openLineId`, `externalId`, and `logoUrl` in `Update.Command` and `Bitrix24Partner` entity. - Update associated tests to cover new validation scenarios.
…s, and refactor `Makefile` and `console` setup.
…aper` services and integrated them into `ScrapePartnersCommandV2` for improved modularity and reusability.
…ndling with new helper methods.
…ScrapePartnersCommandV2: добавлена обработка блокировок, настройки задержек, детектирование пустых страниц и обновлён механизм работы с доменом.
…rtnersCsvCommand, улучшил обработку пустых страниц и блокировок в ScrapePartnersCommandV2, добавил загрузку .env.local.
…nd update `.env` loading in `console`.
…er updates, integrate `ScrapeStateManager` into scraping commands, and refactor CSV handling logic for modularity.
…nsole commands, and refactor partner scraping for improved base domain handling and progress tracking.
…r consistency and improved maintainability across infrastructure, use cases, and console commands.
camaxtly
reviewed
May 10, 2026
| |-------|----------|--------------| | ||
| | `--base-url` | URL страницы списка партнёров | `https://www.bitrix24.ru/partners/country__19/` | | ||
| | `--output-file` | Путь к выходному CSV файлу | `partners.csv` | | ||
| | `--page-delay` | Задержка между страницами (сек) | `2` | |
Collaborator
There was a problem hiding this comment.
Отразить какую проблему можно решить
camaxtly
reviewed
May 10, 2026
| # Кастомный выходной файл и ускоренный парсинг | ||
| php bin/console partners:scrape --output-file=partners_kz.csv --page-delay=1 --partner-delay=1 | ||
|
|
||
| # Продолжить прерванный парсинг |
Collaborator
There was a problem hiding this comment.
Нет, пример команды должен быть полный. Тут хорошо пример запуска и пример продолжения.
camaxtly
reviewed
May 10, 2026
| php bin/console partners:scrape --insecure | ||
| ``` | ||
|
|
||
| **Механизм resume:** При прерывании (Ctrl+C) создаётся state-файл `<output>.state.json` с информацией о последней обработанной странице. При запуске с `--resume` парсинг продолжится с этого места. |
Collaborator
There was a problem hiding this comment.
Поправить про прерывание
camaxtly
reviewed
May 10, 2026
|
|
||
| **Механизм resume:** При прерывании (Ctrl+C) создаётся state-файл `<output>.state.json` с информацией о последней обработанной странице. При запуске с `--resume` парсинг продолжится с этого места. | ||
|
|
||
| **Обнаружение бана:** Команда автоматически определяет блокировку по двум признакам: |
Collaborator
There was a problem hiding this comment.
Бан и продолжение загрузки вынести до команд, как общее описание, что может произойти
camaxtly
reviewed
May 10, 2026
|
|
||
| **Обязательные колонки CSV:** `title`, `bitrix24_partner_number` | ||
|
|
||
| **Опциональные колонки:** `site`, `phone`, `email`, `open_line_id`, `external_id`, `logo_url` |
Collaborator
There was a problem hiding this comment.
Зачем поддерживать кастомную структуру файла?
camaxtly
reviewed
May 10, 2026
| 3240,Hoster.KZ,https://b24.kz/,8-727-2-379-284,info@b24.kz,https://.../logo.jpg,/partners/partner/3240/,https://www.bitrix24.kz,2026-05-01T12:27:22+00:00 | ||
| ``` | ||
|
|
||
| | Колонка | Описание | |
Collaborator
There was a problem hiding this comment.
Эти поля должны быть обязательными при импорте.
camaxtly
requested changes
May 10, 2026
|
|
||
| --- | ||
|
|
||
| ### `partners:update` — Обновление конкретных партнёров |
Collaborator
There was a problem hiding this comment.
Переделываем логику работы ---
- Можно указать по ИД перечень партнеров, которых надо обновить
- Только этих партнеров выгрузаем в файл. Файл и его структура идентична тому, который используется для полной выгрузки.
- Дальше используем процедуру импорта и она обновит данные партнеров из файла.
|
|
||
| --- | ||
|
|
||
| ## Запуск через Docker (Makefile) |
Collaborator
There was a problem hiding this comment.
Актуализровать команды и их описание
|
|
||
| --- | ||
|
|
||
| ## Рекомендации |
Collaborator
There was a problem hiding this comment.
Рекомендации разнести в разделы, как правильно пользоваться либой.
| private const int DEFAULT_PAGE_DELAY = 2; | ||
|
|
||
| private const int DEFAULT_PARTNER_DELAY = 2; | ||
|
|
Collaborator
There was a problem hiding this comment.
Тут не хватает дефолта для insecure
| protected function execute(InputInterface $input, OutputInterface $output): int | ||
| { | ||
| $io = new SymfonyStyle($input, $output); | ||
| $symfonyStyle = new SymfonyStyle($input, $output); |
Collaborator
There was a problem hiding this comment.
назови переменную io, чтоб было одинаково во всех командах, когда ты работает с "разукрашиванием" консоли
| /** @var AggregateRootEventsEmitterInterface|Bitrix24PartnerInterface $activePartner */ | ||
| $activePartner = $this->bitrix24PartnerRepository->findByBitrix24PartnerNumber($command->bitrix24PartnerNumber); | ||
|
|
||
| if (null !== $activePartner) { |
Collaborator
There was a problem hiding this comment.
С таким походом у нас этот юзкейз актуален только для 1 импорта, потом 99% будет отваливаться здесь.
| use libphonenumber\PhoneNumberUtil; | ||
| use Psr\Log\LoggerInterface; | ||
|
|
||
| readonly class Handler |
Collaborator
There was a problem hiding this comment.
Обхединяем с текущим create, так как нам по сути надо обновить данные в БД из файла и все.
А если партнера нет, то его и добавить.
| use Bitrix24\SDK\Application\Contracts\Events\AggregateRootEventsEmitterInterface; | ||
| use Psr\Log\LoggerInterface; | ||
|
|
||
| readonly class Handler |
Collaborator
There was a problem hiding this comment.
Проработать сценарий верхнеуровнено, как и когда будет вызываться этот юзкейс
| @@ -0,0 +1,190 @@ | |||
| # Консольные команды для работы с партнёрами Bitrix24 | |||
|
|
|||
| ## Обзор | |||
Collaborator
There was a problem hiding this comment.
Сделлать еще 1 доку и описать сценарии импорта.
- пропуск,
- обновление данных,
- добавление несуществующего и
- удаление лишнего.
| * | ||
| * @return bool true if the Bitrix24Partner are equal, false otherwise | ||
| */ | ||
| public function equals(Bitrix24PartnerInterface $other): bool |
Collaborator
There was a problem hiding this comment.
Если нужно только для тестов, так и оставь в комментах
…ментацию по сценарию импорта партнёров, переработал ScrapePartnersCommand и UpdatePartnersCommand для синхронизации с новой спецификацией.
- Implement `Command` and `Handler` classes for upsert functionality. - Add validation for required fields in `Upsert\Command`. - Introduce `Upsert\Handler` to handle create, update, or skip logic based on partner data. - Update documentation to include new upsert scenarios and sync modes (`full`, `partial`). - Add unit and functional tests to cover all upsert scenarios, including creation, update, and skip cases. - Refactor partner import logic to align with the new upsert behavior.
…arser dependency: - Move PartnerHtmlParser logic into PartnerPageScraper as a dependency. - Simplify scraper command implementations (`UpdatePartnersCommand`, `ScrapePartnersCommand`) by using the consolidated `fetchPartnerData` method. - Introduce verbosity-based output control in `ScrapePartnersCommand`. - Add `PartnerData` DTO to structure scraped partner data. - Update tests to align with new scraping logic and enhance maintainability.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit implements comprehensive support for Bitrix24Partners following the established DDD/CQRS patterns in the codebase.
Features Implemented:
Core Domain
Use Cases (CQRS)
CLI Tools
Testing
Technical Details:
Resolves: #70