Active Surveillance and Chemotherapy Before Surgery in Treating Participants With Stage II-III Rectal Cancer

Inclusion Criteria:

* Histologically confirmed diagnosis of rectal adenocarcinoma
* Eligible for curative resection of rectal adenocarcinoma
* Rectal tumor location =\< 12 cm from the anal verge as determined by endoscopy or magnetic resonance imaging (MRI) (if endoscopy report is not available or deemed inadequate my treating oncologist)
* Nodal involvement confined to the radiation field
* Radiologically measurable or clinically evaluable disease as defined in the protocol
* Eastern Cooperative Oncology Group (ECOG) performance status (PS): 0, 1 or 2
* Clinical Stage: Stage II and III. N2 disease is to be estimated as four or more lymph nodes that are \>= 10 mm. Clinical staging should be estimated based on the combination of the following assessments: physical exam by the primary surgeon including digital rectal exam (DRE), computed tomography (CT) or positron emission tomography (PET)/CT scan of the chest/abdomen/pelvis and a pelvic MRI. If a pelvic MRI is performed, it is acceptable to perform CT of the chest/abdomen, omitting CT imaging of the pelvis. PET/CT is optional.
* No known contraindication to standard (fluoropyrimidine-based) pelvic chemoradiation (e.g. dihydropyrimidine dehydrogenase \[DPD\] deficiency)
* Patient of child-bearing potential is willing to employ adequate contraception during treatment and after treatment, as directed by treating clinical team
* Willing to provide written informed consent
* Willing to return to enrolling medical site for all study assessments

Exclusion Criteria:

* Diagnosis of inflammatory bowel disease (IBD)
* Diagnosis of MSI-H colorectal cancer at time of consent
* Recurrent rectal cancer
* Tumor is causing symptomatic bowel obstruction (patients who have diverting ostomy are eligible)
* Any prior pelvic radiation
* Other invasive malignancy undergoing active treatment. Patients receiving prior treatment that precludes standard chemoradiation or ability to receive consolidation/adjuvant chemotherapy will be excluded from survival analyses
* Patients unwilling or unable to undergo pelvic MRI