Pancreatic neuroendocrine tumors (PanNETs) are a rare type of pancreatic neoplasm. PanNETs with intraductal growth are considered extremely rare, and ductal involvement in NETs has been reported to correlate with aggressive clinical behavior. A 67-year-old male was referred to our hospital for further evaluation of a pancreatic tumor. Various imaging modalities, including ultrasonography, contrast-enhanced CT, MRI, and endoscopic ultrasonography (EUS), demonstrated a well-defined pancreatic mass in the head and body, primarily located within the main pancreatic duct with extension to the main portal vein. Histological diagnosis from an EUS-guided biopsy revealed a grade 3 well-differentiated NET. The patient was scheduled for surgery; however, multiple additional bone metastases were identified on 18F-fluorodeoxyglucose PET/CT. Although the presence of intraductal growth and portal vein invasion is uncommon in PanNETs, aggressive workup and treatment are essential in such cases.