This research revolves around the modeling and optimization of an innovative geothermal system powered by geothermal energy. The system incorporates a modified Organic Rankine Cycle to efficiently produce electricity, hydrogen, cooling, and heating. The study is specifically tailored for South Korea, with a focus on the city of Ulsan. The primary objective of this geothermal system is to generate essential products for residential buildings without contributing to carbon emissions or environmental pollution. The modeling aspect of this research is conducted using EES software, while optimization efforts are aimed at enhancing performance by optimizing the exergy efficiency and reducing costs. The response surface method (RSM) and Design Expert optimization software are employed for this purpose. The optimal results demonstrate that, in its most efficient state, the system can achieve an impressive exergy efficiency of 45.382 % and operate at a cost rate of 50.414 $/h. The economic analysis underscores that the modified ORC unit bears the highest system cost, amounting to 49.47 $/h. The findings reveal that the suggested geothermal system performs exceptionally well in cities with weather conditions akin to Ulsan. In other regions across the globe, it has the potential to significantly enhance cost-effectiveness and exergy efficiency. The research concludes by evaluating the system's capacity to provide electricity, heating, and cooling for residential complexes within the study areas, highlighting its potential for addressing diverse energy needs in urban settings while prioritizing sustainability and environmental responsibility. The results showed that the study system can produce 3,982,413.6 kWh of heating, and 4,555,440 kWh of cooling during the year. The environmental results showed that by producing 4088.72 MWh of electricity annually in the city of Ulsan, South Korea, it is possible to help the expansion of 4 hectares of green space per year. Also, by reducing carbon dioxide emission by 834.09 tons of CO2, the proposed system avoids the cost of 20,018.3 $ on the environment. The proposed geothermal system can supply the electrical needs of 401 people in Ulsan city during the year.