Peace processes are complex, contentious and dynamic systems which involve significant compromising among various societal and political stakeholders. It becomes pertinent to have a pulse of the institutions as well as the citizens to ensure that the peace process referendum is responsive to the concerns. We posit that social media yields tremendous power in peace processes as a tool for dialogue, organization, and mobilization thereby adding more complexity by opening the peace process to public influence. Using Colombia's peace agreement as a case study, we study two key indicators: inter-group polarization and public sentiment towards the peace process. We present a detailed linguistic analysis to detect inter-group polarization and understand differences in signals emerging from polarized groups. We also present a predictive model which leverages tweet, content and user-based features to predict public sentiment towards the Colombian peace process as observed through social media. We demonstrate that had pro-accord stakeholders leveraged public opinion from social media, they would have been better prepared and the outcome of the Colombian referendum could have been different.