A Pilot Study of Olaparib and Durvalumab in Patients With Metastatic Triple Negative Breast Cancer

• Ability to understand and the willingness to sign a written informed consent document.

• Metastatic triple negative breast cancer (TNBC), as defined by:

‣ Estrogen receptor (ER) and progesterone receptor (PR) negative as defined as ER \< 10% and PR \< 10% by immunohistochemistry according to American Society of Clinical Oncology (ASCO)/College of American Pathologists (CAP) guidelines for hormone receptor testing

⁃ Human epidermal growth factor receptor 2 (HER2) non-amplified per ASCO/CAP guidelines, defined as:

• immunohistochemistry (IHC) score 0/1+

∙ IHC 2+ and in situ hybridization (ISH) non-amplified with a ratio of HER2 to CEP17 \< 2.0, and if reported, average HER2 gene copy number \< 4 signals/cells; or

∙ ISH non-amplified with a ratio of HER2 to chromosome enumeration probe 17 (CEP17) \< 2.0, and if reported, average HER2 gene copy number \< 4 signals/cells

• Participants must have at least one measurable site of disease as defined by Response Evaluation Criteria in Solid Tumors (RECIST) version (v)1.1 that is amendable to biopsy.

• Prior therapies for metastatic breast cancer

‣ Frontline patients who have not received prior systemic therapy for metastatic breast cancer are eligible,

⁃ Patients who have received =\< 2 prior chemotherapy regimens for metastatic breast cancer are eligible

• Participants must have fully recovered from the acute toxic effects of all prior treatment to grade 1 or less, except alopecia and =\< grade 2 neuropathy which are allowed

• Participants' life expectancy must be \> 6 months

• Participant must have Eastern Cooperative Oncology Group (ECOG) performance status =\< 2

• Participant must consent to undergo a pre-treatment screening biopsy for enrollment and subsequent biomarker analyses.

• Participants must consent to undergo one mandatory on-study tumor biopsy following a 4 week, single cycle induction treatment of olaparib. A second on-study biopsy at time of disease progression is optional, but not mandatory.

• Participants must not have had prior immunotherapy with anti-PD-L1, anti-PD-1, anti-CTLA4 or similar drugs.

• Participants must not have received previous treatment with PARP inhibitors

• Participants must not be planning to receive concomitantly other biologic therapy, hormonal therapy, other chemotherapy, surgery or other anti-cancer therapy except radiation therapy while receiving treatment on this protocol.

• Participant must be able to swallow tablets or capsules. A participant with any gastrointestinal disease that would impair ability to swallow, retain, or absorb drug is not eligible.

• Absolute neutrophil count (ANC) \>= 1.5 x 10\^9 /L (at time of registration and within 4 weeks prior to initiating on-protocol treatment)

‣ May be waived on a case-by-case basis for patient populations recognized to have normal baseline values below this level.

• Platelets \>= 100 x 10\^9 /L (at time of registration and within 4 weeks prior to initiating on-protocol treatment)

• Hemoglobin \>= 9 g/dL or \>= 5.6 mmol/L (at time of registration and within 4 weeks prior to initiating on-protocol treatment)

• Creatinine =\< 1.5 x upper limit of normal (ULN), OR measured or calculated creatinine clearance (glomerular filtration rate (GFR) can also be used in place of creatinine or CrCl) \>= 60 mL/min/1.73m\^2 for participants with creatinine levels \> 1 x institutional ULN (at time of registration and within 4 weeks prior to initiating on-protocol treatment)

‣ Creatinine clearance should be calculated per institutional standard. For participants with a baseline calculated creatinine clearance below normal institutional laboratory values, a measured baseline creatinine clearance should be determined. Individuals with higher values felt to be consistent with inborn errors of metabolism will be considered on a case-by-case basis.

• Total bilirubin =\< 1.5 x ULN, OR direct bilirubin =\< ULN for participants with total bilirubin levels \> 1.5 x ULN (at time of registration and within 4 weeks prior to initiating on-protocol treatment)

• Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\]) and alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 3 x ULN (at time of registration and within 4 weeks prior to initiating on-protocol treatment)

• Female participants of childbearing potential must have a negative urine or serum pregnancy test within 72 hours prior to receiving the first dose of study medication. If the urine test is positive or cannot be confirmed as negative, a serum pregnancy test will be required.

• Female participants of childbearing potential must agree to use adequate methods of contraception starting with the first dose of study therapy through 60 days after the last dose of study therapy.

‣ Participants of childbearing potential are those who have not been surgically sterilized or have not been free from menses for \> 1 year without an alternative medical cause.

⁃ Note: Abstinence is acceptable if this is the preferred contraception for the participant.

• Male participants must agree to use an adequate method of contraception starting with the first dose of study therapy through 60 days after the last dose of study therapy.

‣ Note: Abstinence is acceptable if this is the preferred contraception for the participant.

• Participants must not have received live vaccines within 30 days prior to trial registration. Examples of live vaccines include, but are not limited to, the following: measles, mumps, rubella, chicken pox, shingles, yellow fever, rabies, bacillus Calmette-Guerin (BCG), and typhoid (oral) vaccine. Seasonal influenza vaccines for injection are generally killed virus vaccines and are allowed; however, intranasal influenza vaccines (e.g., Flu-Mist) are live attenuated vaccines, and are not allowed.

• Participants must not have known active hepatitis B virus (HBV) or hepatitis C virus (HCV) infection prior to trial registration. Patients who have completed curative therapy for HCV are eligible. Patients with known human immunodeficiency virus (HIV) infection are eligible if they meet each of the following 3 criteria:

‣ CD4 counts \>= 350 mm\^3

⁃ Serum HIV viral load of \< 25,000 IU/ml and

⁃ Treated on a stable antiretroviral regimen.

• No other prior invasive malignancy is allowed except for the following: adequately treated basal (or squamous cell) skin cancer, in situ breast or cervical cancer. Stage I or II invasive cancer treated with a curative intent without evidence of disease recurrence for at least five years.